2023-10-05T20:00:00.065 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-05T20:00:00.065 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/yuriw-2023-10-05_14:31:59-rbd-wip-yuri2-testing-2023-10-04-1333-quincy-distro-default-smithi/7411791 --verbose --description rbd/cli_v1/{base/install clusters/{fixed-1 openstack} conf/{disable-pool-app} features/format-1 msgr-failures/few objectstore/bluestore-comp-zlib pool/small-cache-pool supported-random-distro$/{centos_8} workloads/rbd_cli_generic} --name yuriw-2023-10-05_14:31:59-rbd-wip-yuri2-testing-2023-10-04-1333-quincy-distro-default-smithi --owner scheduled_yuriw@teuthology -- /home/teuthworker/archive/yuriw-2023-10-05_14:31:59-rbd-wip-yuri2-testing-2023-10-04-1333-quincy-distro-default-smithi/7411791/orig.config.yaml 2023-10-05T20:00:00.096 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-05T20:00:00.171 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-10-05_14:31:59-rbd-wip-yuri2-testing-2023-10-04-1333-quincy-distro-default-smithi/7411791 branch: wip-yuri2-testing-2023-10-04-1333-quincy description: rbd/cli_v1/{base/install clusters/{fixed-1 openstack} conf/{disable-pool-app} features/format-1 msgr-failures/few objectstore/bluestore-comp-zlib pool/small-cache-pool supported-random-distro$/{centos_8} workloads/rbd_cli_generic} email: yweinste@redhat.com first_in_suite: false job_id: '7411791' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: yuriw-2023-10-05_14:31:59-rbd-wip-yuri2-testing-2023-10-04-1333-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 3 size: 30 os_type: centos os_version: 8.stream overrides: admin_socket: branch: wip-yuri2-testing-2023-10-04-1333-quincy ceph: conf: client: rbd default format: 1 global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: zlib bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - overall HEALTH_ - \(CACHE_POOL_NEAR_FULL\) - \(CACHE_POOL_NO_HIT_SET\) sha1: 487ffcb5ddb60d29c5688d1a457e812b1b697b90 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log global: osd crush chooseleaf type: 0 osd pool default pg num: 128 osd pool default pgp num: 128 osd pool default size: 2 mon: {} install: ceph: flavor: default sha1: 487ffcb5ddb60d29c5688d1a457e812b1b697b90 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri2-testing-2023-10-04-1333-quincy sha1: 487ffcb5ddb60d29c5688d1a457e812b1b697b90 owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 652 sha1: 487ffcb5ddb60d29c5688d1a457e812b1b697b90 sleep_before_teardown: 0 subset: 111/12000 suite: rbd suite_branch: wip-yuri2-testing-2023-10-04-1333-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_487ffcb5ddb60d29c5688d1a457e812b1b697b90/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 487ffcb5ddb60d29c5688d1a457e812b1b697b90 targets: smithi167.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBIRuzAhk+yMzyhTJFKgQFK82kkX2FDeRQJVhUXDDAz8byAnwnstl+xvDmRzlH0emr8zzGNpH4zXzX5NUQczVBk= tasks: - install: null - ceph: null - exec: client.0: - sudo ceph osd pool create cache 4 - sudo ceph osd tier add rbd cache - sudo ceph osd tier cache-mode cache writeback - sudo ceph osd tier set-overlay rbd cache - sudo ceph osd pool set cache hit_set_type bloom - sudo ceph osd pool set cache hit_set_count 8 - sudo ceph osd pool set cache hit_set_period 60 - sudo ceph osd pool set cache target_max_objects 250 - workunit: clients: client.0: - rbd/cli_generic.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-05_14:31:59 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-05T20:00:00.171 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_487ffcb5ddb60d29c5688d1a457e812b1b697b90/qa; will attempt to use it 2023-10-05T20:00:00.172 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_487ffcb5ddb60d29c5688d1a457e812b1b697b90/qa/tasks 2023-10-05T20:00:00.172 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-05T20:00:00.173 INFO:teuthology.task.internal:Checking packages... 2023-10-05T20:00:00.195 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '487ffcb5ddb60d29c5688d1a457e812b1b697b90' 2023-10-05T20:00:00.195 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-05T20:00:00.195 INFO:teuthology.packaging:ref: None 2023-10-05T20:00:00.195 INFO:teuthology.packaging:tag: None 2023-10-05T20:00:00.195 INFO:teuthology.packaging:branch: wip-yuri2-testing-2023-10-04-1333-quincy 2023-10-05T20:00:00.195 INFO:teuthology.packaging:sha1: 487ffcb5ddb60d29c5688d1a457e812b1b697b90 2023-10-05T20:00:00.196 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri2-testing-2023-10-04-1333-quincy 2023-10-05T20:00:00.419 INFO:teuthology.task.internal:Found packages for ceph version 17.2.6-1278.g487ffcb5 2023-10-05T20:00:00.419 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-05T20:00:00.427 INFO:teuthology.task.internal:no buildpackages task found 2023-10-05T20:00:00.427 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-05T20:00:00.436 INFO:teuthology.task.internal:Saving configuration 2023-10-05T20:00:00.448 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-05T20:00:00.456 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-05T20:00:00.481 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi167.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_14:31:59-rbd-wip-yuri2-testing-2023-10-04-1333-quincy-distro-default-smithi/7411791', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-05 19:54:50.268623', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBIRuzAhk+yMzyhTJFKgQFK82kkX2FDeRQJVhUXDDAz8byAnwnstl+xvDmRzlH0emr8zzGNpH4zXzX5NUQczVBk='} 2023-10-05T20:00:00.481 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-05T20:00:00.488 INFO:teuthology.task.internal:roles: ubuntu@smithi167.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2023-10-05T20:00:00.488 INFO:teuthology.run_tasks:Running task console_log... 2023-10-05T20:00:00.527 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe12afffee0>, signals=[15]) 2023-10-05T20:00:00.528 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-05T20:00:00.539 INFO:teuthology.task.internal:Opening connections... 2023-10-05T20:00:00.539 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi167.front.sepia.ceph.com 2023-10-05T20:00:00.541 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-05T20:00:00.606 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-05T20:00:00.614 DEBUG:teuthology.orchestra.run.smithi167:> uname -m 2023-10-05T20:00:00.680 INFO:teuthology.orchestra.run.smithi167.stdout:x86_64 2023-10-05T20:00:00.681 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release 2023-10-05T20:00:00.741 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="CentOS Stream" 2023-10-05T20:00:00.741 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="8" 2023-10-05T20:00:00.741 INFO:teuthology.orchestra.run.smithi167.stdout:ID="centos" 2023-10-05T20:00:00.741 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="rhel fedora" 2023-10-05T20:00:00.741 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="8" 2023-10-05T20:00:00.741 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el8" 2023-10-05T20:00:00.741 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="CentOS Stream 8" 2023-10-05T20:00:00.741 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31" 2023-10-05T20:00:00.742 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-10-05T20:00:00.742 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://centos.org/" 2023-10-05T20:00:00.742 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-05T20:00:00.742 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-05T20:00:00.742 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-05T20:00:00.742 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server 2023-10-05T20:00:00.766 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-05T20:00:00.776 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-05T20:00:00.789 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-05T20:00:00.789 DEBUG:teuthology.orchestra.run.smithi167:> test '!' -e /home/ubuntu/cephtest 2023-10-05T20:00:00.810 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-05T20:00:00.820 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-05T20:00:00.821 DEBUG:teuthology.orchestra.run.smithi167:> test -z $(ls -A /var/lib/ceph) 2023-10-05T20:00:00.879 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-05T20:00:00.915 INFO:teuthology.run_tasks:Running task kernel... 2023-10-05T20:00:00.930 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-10-05T20:00:00.931 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-10-05T20:00:00.931 DEBUG:teuthology.orchestra.run.smithi167:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-05T20:00:00.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-05T20:00:00.952 DEBUG:teuthology.orchestra.run.smithi167:> uname -r 2023-10-05T20:00:01.014 INFO:teuthology.orchestra.run.smithi167.stdout:4.18.0-489.el8.x86_64 2023-10-05T20:00:01.014 INFO:teuthology.task.kernel:Running kernel on smithi167: 4.18.0-489.el8.x86_64 2023-10-05T20:00:01.014 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum install -y kernel 2023-10-05T20:13:46.358 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 8 - AppStream 42 kB/s | 33 MB 13:42 2023-10-05T20:13:53.334 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 8 - BaseOS 52 MB/s | 48 MB 00:00 2023-10-05T20:14:00.492 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 8 - Extras 59 kB/s | 18 kB 00:00 2023-10-05T20:14:00.817 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 8 - Extras common packages 48 kB/s | 6.9 kB 00:00 2023-10-05T20:14:01.110 INFO:teuthology.orchestra.run.smithi167.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2023-10-05T20:14:02.405 INFO:teuthology.orchestra.run.smithi167.stdout:Extra Packages for Enterprise Linux 14 MB/s | 16 MB 00:01 2023-10-05T20:14:06.228 INFO:teuthology.orchestra.run.smithi167.stdout:lab-extras 422 kB/s | 24 kB 00:00 2023-10-05T20:14:09.811 INFO:teuthology.orchestra.run.smithi167.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2023-10-05T20:14:09.811 INFO:teuthology.orchestra.run.smithi167.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2023-10-05T20:14:10.065 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-10-05T20:14:10.066 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T20:14:10.066 INFO:teuthology.orchestra.run.smithi167.stdout: Package Architecture Version Repository Size 2023-10-05T20:14:10.066 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T20:14:10.066 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2023-10-05T20:14:10.066 INFO:teuthology.orchestra.run.smithi167.stdout: kernel x86_64 4.18.0-514.el8 baseos 10 M 2023-10-05T20:14:10.067 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2023-10-05T20:14:10.067 INFO:teuthology.orchestra.run.smithi167.stdout: kernel-core x86_64 4.18.0-514.el8 baseos 43 M 2023-10-05T20:14:10.067 INFO:teuthology.orchestra.run.smithi167.stdout: kernel-modules x86_64 4.18.0-514.el8 baseos 35 M 2023-10-05T20:14:10.067 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T20:14:10.067 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-10-05T20:14:10.067 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T20:14:10.067 INFO:teuthology.orchestra.run.smithi167.stdout:Install 3 Packages 2023-10-05T20:14:10.067 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T20:14:10.068 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 88 M 2023-10-05T20:14:10.068 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 96 M 2023-10-05T20:14:10.068 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2023-10-05T20:14:11.560 INFO:teuthology.orchestra.run.smithi167.stdout:(1/3): kernel-4.18.0-514.el8.x86_64.rpm 35 MB/s | 10 MB 00:00 2023-10-05T20:14:12.005 INFO:teuthology.orchestra.run.smithi167.stdout:(2/3): kernel-modules-4.18.0-514.el8.x86_64.rpm 48 MB/s | 35 MB 00:00 2023-10-05T20:14:12.429 INFO:teuthology.orchestra.run.smithi167.stdout:(3/3): kernel-core-4.18.0-514.el8.x86_64.rpm 37 MB/s | 43 MB 00:01 2023-10-05T20:14:12.430 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2023-10-05T20:14:12.430 INFO:teuthology.orchestra.run.smithi167.stdout:Total 37 MB/s | 88 MB 00:02 2023-10-05T20:14:13.104 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2023-10-05T20:14:13.539 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2023-10-05T20:14:13.539 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2023-10-05T20:14:14.764 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2023-10-05T20:14:14.792 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2023-10-05T20:14:17.523 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2023-10-05T20:14:18.039 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : kernel-core-4.18.0-514.el8.x86_64 1/3 2023-10-05T20:14:21.613 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: kernel-core-4.18.0-514.el8.x86_64 1/3 2023-10-05T20:14:22.097 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : kernel-modules-4.18.0-514.el8.x86_64 2/3 2023-10-05T20:14:25.474 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: kernel-modules-4.18.0-514.el8.x86_64 2/3 2023-10-05T20:14:25.819 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : kernel-4.18.0-514.el8.x86_64 3/3 2023-10-05T20:14:47.747 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: kernel-core-4.18.0-514.el8.x86_64 3/3 2023-10-05T20:14:47.772 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: kernel-modules-4.18.0-514.el8.x86_64 3/3 2023-10-05T20:14:48.134 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: kernel-4.18.0-514.el8.x86_64 3/3 2023-10-05T20:14:48.134 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : kernel-4.18.0-514.el8.x86_64 1/3 2023-10-05T20:14:48.134 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : kernel-core-4.18.0-514.el8.x86_64 2/3 2023-10-05T20:14:49.159 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : kernel-modules-4.18.0-514.el8.x86_64 3/3 2023-10-05T20:14:49.159 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T20:14:49.159 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2023-10-05T20:14:49.159 INFO:teuthology.orchestra.run.smithi167.stdout: kernel-4.18.0-514.el8.x86_64 kernel-core-4.18.0-514.el8.x86_64 2023-10-05T20:14:49.160 INFO:teuthology.orchestra.run.smithi167.stdout: kernel-modules-4.18.0-514.el8.x86_64 2023-10-05T20:14:49.160 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T20:14:49.160 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-10-05T20:14:49.352 DEBUG:teuthology.orchestra.run.smithi167:> rpm -q kernel | sort -rV | head -n 1 2023-10-05T20:14:49.608 INFO:teuthology.orchestra.run.smithi167.stdout:kernel-4.18.0-514.el8.x86_64 2023-10-05T20:14:49.609 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-514.el8.x86_64 2023-10-05T20:14:49.609 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-514.el8.x86_64 2023-10-05T20:14:49.609 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-05T20:14:49.609 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-05T20:14:49.609 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-10-05T20:14:49.609 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi167.front.sepia.ceph.com, path=None, version=distro) 2023-10-05T20:14:49.609 DEBUG:teuthology.orchestra.run.smithi167:> rpm -q kernel | sort -rV | head -n 1 2023-10-05T20:14:49.817 INFO:teuthology.orchestra.run.smithi167.stdout:kernel-4.18.0-514.el8.x86_64 2023-10-05T20:14:49.817 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-514.el8.x86_64 2023-10-05T20:14:49.817 DEBUG:teuthology.orchestra.run.smithi167:> sudo rpm -qi grub2-tools 2023-10-05T20:14:49.851 INFO:teuthology.orchestra.run.smithi167.stdout:Name : grub2-tools 2023-10-05T20:14:49.851 INFO:teuthology.orchestra.run.smithi167.stdout:Epoch : 1 2023-10-05T20:14:49.851 INFO:teuthology.orchestra.run.smithi167.stdout:Version : 2.02 2023-10-05T20:14:49.852 INFO:teuthology.orchestra.run.smithi167.stdout:Release : 129.el8 2023-10-05T20:14:49.852 INFO:teuthology.orchestra.run.smithi167.stdout:Architecture: x86_64 2023-10-05T20:14:49.852 INFO:teuthology.orchestra.run.smithi167.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-10-05T20:14:49.852 INFO:teuthology.orchestra.run.smithi167.stdout:Group : System Environment/Base 2023-10-05T20:14:49.852 INFO:teuthology.orchestra.run.smithi167.stdout:Size : 9265586 2023-10-05T20:14:49.852 INFO:teuthology.orchestra.run.smithi167.stdout:License : GPLv3+ 2023-10-05T20:14:49.852 INFO:teuthology.orchestra.run.smithi167.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-10-05T20:14:49.852 INFO:teuthology.orchestra.run.smithi167.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-10-05T20:14:49.852 INFO:teuthology.orchestra.run.smithi167.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-10-05T20:14:49.852 INFO:teuthology.orchestra.run.smithi167.stdout:Build Host : kbuilder.bsys.centos.org 2023-10-05T20:14:49.852 INFO:teuthology.orchestra.run.smithi167.stdout:Relocations : (not relocatable) 2023-10-05T20:14:49.852 INFO:teuthology.orchestra.run.smithi167.stdout:Packager : CentOS BuildSystem 2023-10-05T20:14:49.853 INFO:teuthology.orchestra.run.smithi167.stdout:Vendor : CentOS 2023-10-05T20:14:49.853 INFO:teuthology.orchestra.run.smithi167.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-05T20:14:49.853 INFO:teuthology.orchestra.run.smithi167.stdout:Summary : Support tools for GRUB. 2023-10-05T20:14:49.853 INFO:teuthology.orchestra.run.smithi167.stdout:Description : 2023-10-05T20:14:49.853 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T20:14:49.853 INFO:teuthology.orchestra.run.smithi167.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-05T20:14:49.853 INFO:teuthology.orchestra.run.smithi167.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-05T20:14:49.853 INFO:teuthology.orchestra.run.smithi167.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-05T20:14:49.853 INFO:teuthology.orchestra.run.smithi167.stdout:hardware devices. 2023-10-05T20:14:49.853 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T20:14:49.853 INFO:teuthology.orchestra.run.smithi167.stdout:This subpackage provides tools for support of all platforms. 2023-10-05T20:14:49.854 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-05T20:14:49.854 INFO:teuthology.task.kernel:Updating grub on smithi167 to boot 4.18.0-514.el8.x86_64 2023-10-05T20:14:49.854 DEBUG:teuthology.orchestra.run.smithi167:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-05T20:14:51.284 INFO:teuthology.orchestra.run.smithi167.stderr:Generating grub configuration file ... 2023-10-05T20:14:53.839 INFO:teuthology.orchestra.run.smithi167.stderr:done 2023-10-05T20:14:54.506 DEBUG:teuthology.orchestra.run.smithi167:> mktemp 2023-10-05T20:14:54.550 INFO:teuthology.orchestra.run.smithi167.stdout:/tmp/tmp.h7W881pFeU 2023-10-05T20:14:54.551 DEBUG:teuthology.orchestra.run.smithi167:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.h7W881pFeU 2023-10-05T20:14:54.627 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 0666 /tmp/tmp.h7W881pFeU 2023-10-05T20:14:55.159 DEBUG:teuthology.orchestra.remote:smithi167:/tmp/tmp.h7W881pFeU is 6KB 2023-10-05T20:14:55.184 DEBUG:teuthology.orchestra.run.smithi167:> rm -fr /tmp/tmp.h7W881pFeU 2023-10-05T20:14:55.202 DEBUG:teuthology.orchestra.run.smithi167:> sudo /bin/ls /boot/loader/entries || true 2023-10-05T20:14:55.282 INFO:teuthology.orchestra.run.smithi167.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-10-05T20:14:55.282 INFO:teuthology.orchestra.run.smithi167.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-10-05T20:14:55.282 INFO:teuthology.orchestra.run.smithi167.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-10-05T20:14:55.282 INFO:teuthology.orchestra.run.smithi167.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-514.el8.x86_64.conf 2023-10-05T20:14:55.283 DEBUG:teuthology.orchestra.run.smithi167:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-514.el8.x86_64 2023-10-05T20:14:55.443 DEBUG:teuthology.orchestra.run.smithi167:> sudo shutdown -r now 2023-10-05T20:15:25.483 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-10-05T20:15:25.484 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2023-10-05T20:15:25.485 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-05T20:16:00.122 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.167 2023-10-05T20:16:09.130 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2023-10-05T20:16:09.131 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-05T20:16:27.514 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.167 2023-10-05T20:16:39.526 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2023-10-05T20:16:39.527 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-05T20:16:39.908 DEBUG:teuthology.orchestra.run.smithi167:> true 2023-10-05T20:16:40.338 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2023-10-05T20:16:40.338 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.18.0-514.el8.x86_64"... 2023-10-05T20:16:40.339 DEBUG:teuthology.orchestra.run.smithi167:> uname -r 2023-10-05T20:16:40.356 INFO:teuthology.orchestra.run.smithi167.stdout:4.18.0-514.el8.x86_64 2023-10-05T20:16:40.356 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-514.el8.x86_64 vs 4.18.0-514.el8.x86_64 2023-10-05T20:16:40.356 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-05T20:16:40.356 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-10-05T20:16:41.357 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-10-05T20:16:41.358 DEBUG:teuthology.orchestra.run.smithi167:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-05T20:16:41.445 INFO:teuthology.orchestra.run.smithi167.stdout:ttyS1 2023-10-05T20:16:41.457 DEBUG:teuthology.parallel:result is None 2023-10-05T20:16:41.458 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-05T20:16:41.465 INFO:teuthology.task.internal:Creating test directory... 2023-10-05T20:16:41.465 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-05T20:16:41.482 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-05T20:16:41.489 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-05T20:16:41.495 INFO:teuthology.task.internal:Creating archive directory... 2023-10-05T20:16:41.495 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-05T20:16:41.563 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-05T20:16:41.570 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-05T20:16:41.570 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-05T20:16:41.639 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-05T20:16:41.650 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-05T20:16:41.652 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-05T20:16:41.659 INFO:teuthology.task.internal:Configuring sudo... 2023-10-05T20:16:41.659 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-05T20:16:41.720 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-05T20:16:41.729 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-05T20:16:41.729 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-05T20:16:41.778 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-05T20:16:41.866 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-05T20:16:41.909 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-05T20:16:41.994 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-05T20:16:42.036 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-05T20:16:42.036 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-05T20:16:42.104 DEBUG:teuthology.orchestra.run.smithi167:> sudo service rsyslog restart 2023-10-05T20:16:42.219 INFO:teuthology.orchestra.run.smithi167.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-05T20:16:42.531 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-05T20:16:42.541 INFO:teuthology.task.internal:Starting timer... 2023-10-05T20:16:42.541 INFO:teuthology.run_tasks:Running task pcp... 2023-10-05T20:16:42.585 INFO:teuthology.run_tasks:Running task selinux... 2023-10-05T20:16:42.608 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-10-05T20:16:42.609 DEBUG:teuthology.orchestra.run.smithi167:> sudo service auditd rotate 2023-10-05T20:16:42.699 INFO:teuthology.orchestra.run.smithi167.stdout:Rotating logs: 2023-10-05T20:16:42.701 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-05T20:16:42.701 DEBUG:teuthology.orchestra.run.smithi167:> /usr/sbin/getenforce 2023-10-05T20:16:42.726 INFO:teuthology.orchestra.run.smithi167.stdout:Permissive 2023-10-05T20:16:42.726 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi167.front.sepia.ceph.com': 'permissive'} 2023-10-05T20:16:42.726 DEBUG:teuthology.orchestra.run.smithi167:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-05T20:16:42.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-05T20:16:42.794 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-05T20:16:42.795 DEBUG:teuthology.orchestra.run.smithi167:> sudo /usr/sbin/setenforce permissive 2023-10-05T20:16:42.863 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-05T20:16:42.875 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-10-05T20:16:42.876 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-05T20:16:42.892 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-05T20:16:42.893 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi167.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-05T20:42:11.803 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi167.front.sepia.ceph.com')] 2023-10-05T20:42:11.804 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2023-10-05T20:42:11.805 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-05T20:42:11.873 DEBUG:teuthology.orchestra.run.smithi167:> true 2023-10-05T20:42:11.941 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2023-10-05T20:42:11.942 INFO:teuthology.run_tasks:Running task clock... 2023-10-05T20:42:11.952 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-05T20:42:11.952 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-05T20:42:11.953 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-05T20:42:12.014 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-05T20:42:12.028 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-05T20:42:12.098 INFO:teuthology.orchestra.run.smithi167.stderr:sudo: ntpd: command not found 2023-10-05T20:42:12.109 INFO:teuthology.orchestra.run.smithi167.stdout:506 Cannot talk to daemon 2023-10-05T20:42:12.123 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-05T20:42:12.137 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-05T20:42:12.211 INFO:teuthology.orchestra.run.smithi167.stderr:bash: ntpq: command not found 2023-10-05T20:42:12.214 INFO:teuthology.orchestra.run.smithi167.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-05T20:42:12.214 INFO:teuthology.orchestra.run.smithi167.stdout:=============================================================================== 2023-10-05T20:42:12.214 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-05T20:42:12.214 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-05T20:42:12.214 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-05T20:42:12.214 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-05T20:42:12.215 INFO:teuthology.run_tasks:Running task install... 2023-10-05T20:42:12.224 DEBUG:teuthology.task.install:project ceph 2023-10-05T20:42:12.224 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '487ffcb5ddb60d29c5688d1a457e812b1b697b90'}} 2023-10-05T20:42:12.224 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '487ffcb5ddb60d29c5688d1a457e812b1b697b90'} 2023-10-05T20:42:12.224 INFO:teuthology.task.install:Using flavor: default 2023-10-05T20:42:12.230 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-05T20:42:12.230 INFO:teuthology.task.install:extra packages: [] 2023-10-05T20:42:12.230 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '487ffcb5ddb60d29c5688d1a457e812b1b697b90', 'tag': None, 'wait_for_package': False} 2023-10-05T20:42:12.231 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=487ffcb5ddb60d29c5688d1a457e812b1b697b90 2023-10-05T20:42:12.453 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/centos/8/flavors/default/ 2023-10-05T20:42:12.453 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1278.g487ffcb5 2023-10-05T20:42:12.565 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-05T20:42:12.565 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-05T20:42:12.565 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-05T20:42:12.595 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-10-05T20:42:12.595 DEBUG:teuthology.orchestra.run.smithi167:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/487ffcb5ddb60d29c5688d1a457e812b1b697b90/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-05T20:42:12.663 DEBUG:teuthology.orchestra.run.smithi167:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-05T20:42:12.741 DEBUG:teuthology.orchestra.run.smithi167:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-05T20:42:12.808 INFO:teuthology.orchestra.run.smithi167.stdout:check_obsoletes = 1 2023-10-05T20:42:12.809 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum clean all 2023-10-05T20:42:13.077 INFO:teuthology.orchestra.run.smithi167.stdout:76 files removed 2023-10-05T20:42:13.106 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-radosgw 2023-10-05T20:42:14.075 INFO:teuthology.orchestra.run.smithi167.stdout:ceph packages for x86_64 140 kB/s | 78 kB 00:00 2023-10-05T20:42:14.560 INFO:teuthology.orchestra.run.smithi167.stdout:ceph noarch packages 21 kB/s | 9.2 kB 00:00 2023-10-05T20:42:14.819 INFO:teuthology.orchestra.run.smithi167.stdout:ceph source packages 6.3 kB/s | 1.4 kB 00:00 2023-10-05T20:52:19.344 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 8 - AppStream 56 kB/s | 33 MB 10:04 2023-10-05T20:52:26.219 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 8 - BaseOS 60 MB/s | 48 MB 00:00 2023-10-05T20:52:33.577 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 8 - Extras 38 kB/s | 18 kB 00:00 2023-10-05T20:52:33.903 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 8 - Extras common packages 46 kB/s | 6.9 kB 00:00 2023-10-05T20:52:34.862 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 8 - PowerTools 8.0 MB/s | 6.5 MB 00:00 2023-10-05T20:52:36.273 INFO:teuthology.orchestra.run.smithi167.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2023-10-05T20:52:37.172 INFO:teuthology.orchestra.run.smithi167.stdout:Extra Packages for Enterprise Linux 21 MB/s | 16 MB 00:00 2023-10-05T20:52:40.974 INFO:teuthology.orchestra.run.smithi167.stdout:lab-extras 344 kB/s | 24 kB 00:00 2023-10-05T20:52:45.373 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-10-05T20:52:45.375 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T20:52:45.375 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2023-10-05T20:52:45.375 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T20:52:45.376 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2023-10-05T20:52:45.376 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw x86_64 2:17.2.6-1278.g487ffcb5.el8 ceph 17 M 2023-10-05T20:52:45.376 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrading: 2023-10-05T20:52:45.376 INFO:teuthology.orchestra.run.smithi167.stdout: librados2 x86_64 2:17.2.6-1278.g487ffcb5.el8 ceph 3.5 M 2023-10-05T20:52:45.376 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1 x86_64 2:17.2.6-1278.g487ffcb5.el8 ceph 3.6 M 2023-10-05T20:52:45.376 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2023-10-05T20:52:45.377 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base x86_64 2:17.2.6-1278.g487ffcb5.el8 ceph 5.2 M 2023-10-05T20:52:45.377 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common x86_64 2:17.2.6-1278.g487ffcb5.el8 ceph 23 M 2023-10-05T20:52:45.377 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux x86_64 2:17.2.6-1278.g487ffcb5.el8 ceph 24 k 2023-10-05T20:52:45.377 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-05T20:52:45.377 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2 x86_64 2:17.2.6-1278.g487ffcb5.el8 ceph 797 k 2023-10-05T20:52:45.377 INFO:teuthology.orchestra.run.smithi167.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-05T20:52:45.378 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq x86_64 0.9.0-4.el8 baseos 47 k 2023-10-05T20:52:45.378 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1 x86_64 2:17.2.6-1278.g487ffcb5.el8 ceph 506 k 2023-10-05T20:52:45.378 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka x86_64 0.11.4-3.el8 appstream 354 k 2023-10-05T20:52:45.378 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2 x86_64 2:17.2.6-1278.g487ffcb5.el8 ceph 5.6 M 2023-10-05T20:52:45.378 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-05T20:52:45.378 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2023-10-05T20:52:45.378 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse x86_64 2:17.2.6-1278.g487ffcb5.el8 ceph 45 k 2023-10-05T20:52:45.379 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common x86_64 2:17.2.6-1278.g487ffcb5.el8 ceph 115 k 2023-10-05T20:52:45.379 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs x86_64 2:17.2.6-1278.g487ffcb5.el8 ceph 191 k 2023-10-05T20:52:45.379 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2023-10-05T20:52:45.379 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados x86_64 2:17.2.6-1278.g487ffcb5.el8 ceph 370 k 2023-10-05T20:52:45.379 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd x86_64 2:17.2.6-1278.g487ffcb5.el8 ceph 359 k 2023-10-05T20:52:45.379 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw x86_64 2:17.2.6-1278.g487ffcb5.el8 ceph 106 k 2023-10-05T20:52:45.379 INFO:teuthology.orchestra.run.smithi167.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-05T20:52:45.379 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T20:52:45.380 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-10-05T20:52:45.380 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T20:52:45.380 INFO:teuthology.orchestra.run.smithi167.stdout:Install 21 Packages 2023-10-05T20:52:45.380 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrade 2 Packages 2023-10-05T20:52:45.380 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T20:52:45.381 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 63 M 2023-10-05T20:52:45.381 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2023-10-05T20:52:46.535 INFO:teuthology.orchestra.run.smithi167.stdout:(1/23): ceph-base-17.2.6-1278.g487ffcb5.el8.x86 5.7 MB/s | 5.2 MB 00:00 2023-10-05T20:52:46.642 INFO:teuthology.orchestra.run.smithi167.stdout:(2/23): ceph-selinux-17.2.6-1278.g487ffcb5.el8. 227 kB/s | 24 kB 00:00 2023-10-05T20:52:47.087 INFO:teuthology.orchestra.run.smithi167.stdout:(3/23): libcephfs2-17.2.6-1278.g487ffcb5.el8.x8 1.7 MB/s | 797 kB 00:00 2023-10-05T20:52:47.226 INFO:teuthology.orchestra.run.smithi167.stdout:(4/23): libradosstriper1-17.2.6-1278.g487ffcb5. 3.6 MB/s | 506 kB 00:00 2023-10-05T20:52:47.809 INFO:teuthology.orchestra.run.smithi167.stdout:(5/23): ceph-radosgw-17.2.6-1278.g487ffcb5.el8. 7.7 MB/s | 17 MB 00:02 2023-10-05T20:52:48.051 INFO:teuthology.orchestra.run.smithi167.stdout:(6/23): ceph-common-17.2.6-1278.g487ffcb5.el8.x 9.7 MB/s | 23 MB 00:02 2023-10-05T20:52:48.076 INFO:teuthology.orchestra.run.smithi167.stdout:(7/23): python3-ceph-argparse-17.2.6-1278.g487f 167 kB/s | 45 kB 00:00 2023-10-05T20:52:48.102 INFO:teuthology.orchestra.run.smithi167.stdout:(8/23): python3-ceph-common-17.2.6-1278.g487ffc 2.3 MB/s | 115 kB 00:00 2023-10-05T20:52:48.177 INFO:teuthology.orchestra.run.smithi167.stdout:(9/23): librgw2-17.2.6-1278.g487ffcb5.el8.x86_6 5.8 MB/s | 5.6 MB 00:00 2023-10-05T20:52:48.203 INFO:teuthology.orchestra.run.smithi167.stdout:(10/23): python3-cephfs-17.2.6-1278.g487ffcb5.e 1.5 MB/s | 191 kB 00:00 2023-10-05T20:52:48.286 INFO:teuthology.orchestra.run.smithi167.stdout:(11/23): python3-rados-17.2.6-1278.g487ffcb5.el 2.0 MB/s | 370 kB 00:00 2023-10-05T20:52:48.311 INFO:teuthology.orchestra.run.smithi167.stdout:(12/23): python3-rgw-17.2.6-1278.g487ffcb5.el8. 982 kB/s | 106 kB 00:00 2023-10-05T20:52:48.337 INFO:teuthology.orchestra.run.smithi167.stdout:(13/23): python3-rbd-17.2.6-1278.g487ffcb5.el8. 2.2 MB/s | 359 kB 00:00 2023-10-05T20:57:48.338 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] librdkafka-0.11.4-3.el8.x86_64.rpm: Curl error (28): Timeout was reached for http://mirrors.cmich.edu/centos/8-stream/AppStream/x86_64/os/Packages/librdkafka-0.11.4-3.el8.x86_64.rpm [Operation too slow. Less than 1000 bytes/sec transferred the last 300 seconds] 2023-10-05T20:57:48.844 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] lttng-ust-2.8.1-11.el8.x86_64.rpm: Curl error (28): Timeout was reached for http://mirrors.cmich.edu/centos/8-stream/AppStream/x86_64/os/Packages/lttng-ust-2.8.1-11.el8.x86_64.rpm [Operation too slow. Less than 1000 bytes/sec transferred the last 300 seconds] 2023-10-05T20:57:48.844 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] python3-prettytable-0.7.2-14.el8.noarch.rpm: Curl error (28): Timeout was reached for http://mirrors.cmich.edu/centos/8-stream/AppStream/x86_64/os/Packages/python3-prettytable-0.7.2-14.el8.noarch.rpm [Operation too slow. Less than 1000 bytes/sec transferred the last 300 seconds] 2023-10-05T20:57:49.271 INFO:teuthology.orchestra.run.smithi167.stdout:(14/23): python3-prettytable-0.7.2-14.el8.noarc 149 B/s | 44 kB 05:00 2023-10-05T20:57:49.329 INFO:teuthology.orchestra.run.smithi167.stdout:(15/23): librabbitmq-0.9.0-4.el8.x86_64.rpm 810 kB/s | 47 kB 00:00 2023-10-05T20:57:49.513 INFO:teuthology.orchestra.run.smithi167.stdout:(16/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 882 B/s | 259 kB 05:01 2023-10-05T20:57:49.538 INFO:teuthology.orchestra.run.smithi167.stdout:(17/23): librdkafka-0.11.4-3.el8.x86_64.rpm 1.2 kB/s | 354 kB 05:01 2023-10-05T20:57:49.805 INFO:teuthology.orchestra.run.smithi167.stdout:(18/23): gperftools-libs-2.7-9.el8.x86_64.rpm 644 kB/s | 306 kB 00:00 2023-10-05T20:57:49.830 INFO:teuthology.orchestra.run.smithi167.stdout:(19/23): liboath-2.6.2-3.el8.x86_64.rpm 186 kB/s | 59 kB 00:00 2023-10-05T20:57:49.855 INFO:teuthology.orchestra.run.smithi167.stdout:(20/23): libunwind-1.3.1-3.el8.x86_64.rpm 237 kB/s | 75 kB 00:00 2023-10-05T20:57:50.048 INFO:teuthology.orchestra.run.smithi167.stdout:(21/23): thrift-0.13.0-2.el8.x86_64.rpm 7.2 MB/s | 1.7 MB 00:00 2023-10-05T20:57:50.248 INFO:teuthology.orchestra.run.smithi167.stdout:(22/23): librbd1-17.2.6-1278.g487ffcb5.el8.x86_ 9.1 MB/s | 3.6 MB 00:00 2023-10-05T20:57:50.515 INFO:teuthology.orchestra.run.smithi167.stdout:(23/23): librados2-17.2.6-1278.g487ffcb5.el8.x8 5.1 MB/s | 3.5 MB 00:00 2023-10-05T20:57:50.519 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2023-10-05T20:57:50.520 INFO:teuthology.orchestra.run.smithi167.stdout:Total 212 kB/s | 63 MB 05:05 2023-10-05T20:57:50.559 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2023-10-05T20:57:50.605 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2023-10-05T20:57:50.605 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2023-10-05T20:57:51.000 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2023-10-05T20:57:51.003 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2023-10-05T20:57:52.092 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2023-10-05T20:57:52.293 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/25 2023-10-05T20:57:52.466 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2023-10-05T20:57:52.857 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2023-10-05T20:57:53.033 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librados2-2:17.2.6-1278.g487ffcb5.el8.x86_64 3/25 2023-10-05T20:57:53.132 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-2:17.2.6-1278.g487ffcb5.el8.x86_64 3/25 2023-10-05T20:57:53.424 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rados-2:17.2.6-1278.g487ffcb5.el8.x86_64 4/25 2023-10-05T20:57:53.608 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs2-2:17.2.6-1278.g487ffcb5.el8.x86_64 5/25 2023-10-05T20:57:54.016 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephfs2-2:17.2.6-1278.g487ffcb5.el8.x86_64 5/25 2023-10-05T20:57:54.174 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librbd1-2:17.2.6-1278.g487ffcb5.el8.x86_64 6/25 2023-10-05T20:57:54.380 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-2:17.2.6-1278.g487ffcb5.el8.x86_64 6/25 2023-10-05T20:57:54.571 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : thrift-0.13.0-2.el8.x86_64 7/25 2023-10-05T20:57:54.739 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 8/25 2023-10-05T20:57:54.924 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 9/25 2023-10-05T20:57:55.481 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 9/25 2023-10-05T20:57:55.627 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librgw2-2:17.2.6-1278.g487ffcb5.el8.x86_64 10/25 2023-10-05T20:57:55.693 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librgw2-2:17.2.6-1278.g487ffcb5.el8.x86_64 10/25 2023-10-05T20:57:55.851 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-argparse-2:17.2.6-1278.g487ffcb5.el8. 11/25 2023-10-05T20:57:56.028 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cephfs-2:17.2.6-1278.g487ffcb5.el8.x86_64 12/25 2023-10-05T20:57:56.268 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rgw-2:17.2.6-1278.g487ffcb5.el8.x86_64 13/25 2023-10-05T20:57:56.486 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rbd-2:17.2.6-1278.g487ffcb5.el8.x86_64 14/25 2023-10-05T20:57:56.641 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libradosstriper1-2:17.2.6-1278.g487ffcb5.el8.x86_6 15/25 2023-10-05T20:57:56.707 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1278.g487ffcb5.el8.x86_6 15/25 2023-10-05T20:57:56.909 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 16/25 2023-10-05T20:57:57.091 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 17/25 2023-10-05T20:57:57.266 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2023-10-05T20:57:57.458 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-common-2:17.2.6-1278.g487ffcb5.el8.x8 19/25 2023-10-05T20:58:00.133 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:17.2.6-1278.g487ffcb5.el8.x86_64 20/25 2023-10-05T20:58:00.349 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-common-2:17.2.6-1278.g487ffcb5.el8.x86_64 20/25 2023-10-05T20:58:00.930 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:17.2.6-1278.g487ffcb5.el8.x86_64 20/25 2023-10-05T20:58:01.154 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-base-2:17.2.6-1278.g487ffcb5.el8.x86_64 21/25 2023-10-05T20:58:01.264 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-base-2:17.2.6-1278.g487ffcb5.el8.x86_64 21/25 2023-10-05T20:58:01.415 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-selinux-2:17.2.6-1278.g487ffcb5.el8.x86_64 22/25 2023-10-05T20:58:19.853 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1278.g487ffcb5.el8.x86_64 22/25 2023-10-05T20:58:19.853 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /sys 2023-10-05T20:58:19.854 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /proc 2023-10-05T20:58:19.854 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /dev 2023-10-05T20:58:19.854 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /run 2023-10-05T20:58:19.854 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /mnt 2023-10-05T20:58:19.854 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /var/tmp 2023-10-05T20:58:19.854 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /home 2023-10-05T20:58:19.854 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /tmp 2023-10-05T20:58:19.854 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /dev 2023-10-05T20:58:19.854 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T20:58:21.551 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-radosgw-2:17.2.6-1278.g487ffcb5.el8.x86_64 23/25 2023-10-05T20:58:21.632 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1278.g487ffcb5.el8.x86_64 23/25 2023-10-05T20:58:21.632 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-10-05T20:58:21.831 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-10-05T20:58:21.831 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-10-05T20:58:22.528 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-10-05T20:58:22.528 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-base-2:17.2.6-1278.g487ffcb5.el8.x86_64 1/25 2023-10-05T20:58:22.528 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-common-2:17.2.6-1278.g487ffcb5.el8.x86_64 2/25 2023-10-05T20:58:22.528 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-radosgw-2:17.2.6-1278.g487ffcb5.el8.x86_64 3/25 2023-10-05T20:58:22.528 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-selinux-2:17.2.6-1278.g487ffcb5.el8.x86_64 4/25 2023-10-05T20:58:22.528 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs2-2:17.2.6-1278.g487ffcb5.el8.x86_64 5/25 2023-10-05T20:58:22.528 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libradosstriper1-2:17.2.6-1278.g487ffcb5.el8.x86_6 6/25 2023-10-05T20:58:22.529 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librgw2-2:17.2.6-1278.g487ffcb5.el8.x86_64 7/25 2023-10-05T20:58:22.529 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1278.g487ffcb5.el8. 8/25 2023-10-05T20:58:22.529 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-common-2:17.2.6-1278.g487ffcb5.el8.x8 9/25 2023-10-05T20:58:22.529 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cephfs-2:17.2.6-1278.g487ffcb5.el8.x86_64 10/25 2023-10-05T20:58:22.529 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rados-2:17.2.6-1278.g487ffcb5.el8.x86_64 11/25 2023-10-05T20:58:22.529 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rbd-2:17.2.6-1278.g487ffcb5.el8.x86_64 12/25 2023-10-05T20:58:22.529 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rgw-2:17.2.6-1278.g487ffcb5.el8.x86_64 13/25 2023-10-05T20:58:22.529 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 14/25 2023-10-05T20:58:22.529 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2023-10-05T20:58:22.529 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2023-10-05T20:58:22.529 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 17/25 2023-10-05T20:58:22.529 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2023-10-05T20:58:22.530 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2023-10-05T20:58:22.530 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2023-10-05T20:58:22.530 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2023-10-05T20:58:22.530 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-2:17.2.6-1278.g487ffcb5.el8.x86_64 22/25 2023-10-05T20:58:22.530 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-10-05T20:58:22.530 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-2:17.2.6-1278.g487ffcb5.el8.x86_64 24/25 2023-10-05T20:58:23.582 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-10-05T20:58:23.582 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T20:58:23.582 INFO:teuthology.orchestra.run.smithi167.stdout:Upgraded: 2023-10-05T20:58:23.582 INFO:teuthology.orchestra.run.smithi167.stdout: librados2-2:17.2.6-1278.g487ffcb5.el8.x86_64 2023-10-05T20:58:23.582 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1-2:17.2.6-1278.g487ffcb5.el8.x86_64 2023-10-05T20:58:23.582 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2023-10-05T20:58:23.583 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base-2:17.2.6-1278.g487ffcb5.el8.x86_64 2023-10-05T20:58:23.583 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common-2:17.2.6-1278.g487ffcb5.el8.x86_64 2023-10-05T20:58:23.583 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw-2:17.2.6-1278.g487ffcb5.el8.x86_64 2023-10-05T20:58:23.583 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux-2:17.2.6-1278.g487ffcb5.el8.x86_64 2023-10-05T20:58:23.583 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-05T20:58:23.583 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2-2:17.2.6-1278.g487ffcb5.el8.x86_64 2023-10-05T20:58:23.583 INFO:teuthology.orchestra.run.smithi167.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-05T20:58:23.583 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-10-05T20:58:23.583 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1-2:17.2.6-1278.g487ffcb5.el8.x86_64 2023-10-05T20:58:23.583 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-05T20:58:23.584 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2-2:17.2.6-1278.g487ffcb5.el8.x86_64 2023-10-05T20:58:23.584 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-05T20:58:23.584 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-05T20:58:23.584 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse-2:17.2.6-1278.g487ffcb5.el8.x86_64 2023-10-05T20:58:23.584 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common-2:17.2.6-1278.g487ffcb5.el8.x86_64 2023-10-05T20:58:23.584 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs-2:17.2.6-1278.g487ffcb5.el8.x86_64 2023-10-05T20:58:23.584 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-05T20:58:23.584 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados-2:17.2.6-1278.g487ffcb5.el8.x86_64 2023-10-05T20:58:23.584 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd-2:17.2.6-1278.g487ffcb5.el8.x86_64 2023-10-05T20:58:23.584 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw-2:17.2.6-1278.g487ffcb5.el8.x86_64 2023-10-05T20:58:23.584 INFO:teuthology.orchestra.run.smithi167.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-05T20:58:23.585 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T20:58:23.585 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-10-05T20:58:23.793 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-test 2023-10-05T20:58:24.647 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:05:44 ago on Thu 05 Oct 2023 08:52:40 PM UTC. 2023-10-05T20:58:26.241 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-10-05T20:58:26.243 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T20:58:26.243 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2023-10-05T20:58:26.243 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T20:58:26.243 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2023-10-05T20:58:26.243 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test x86_64 2:17.2.6-1278.g487ffcb5.el8 ceph 52 M 2023-10-05T20:58:26.243 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2023-10-05T20:58:26.243 INFO:teuthology.orchestra.run.smithi167.stdout: jq x86_64 1.6-7.el8 appstream 202 k 2023-10-05T20:58:26.243 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite x86_64 2:17.2.6-1278.g487ffcb5.el8 ceph 174 k 2023-10-05T20:58:26.243 INFO:teuthology.orchestra.run.smithi167.stdout: oniguruma x86_64 6.8.2-2.el8 appstream 187 k 2023-10-05T20:58:26.244 INFO:teuthology.orchestra.run.smithi167.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2023-10-05T20:58:26.244 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2023-10-05T20:58:26.244 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T20:58:26.244 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-10-05T20:58:26.244 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T20:58:26.244 INFO:teuthology.orchestra.run.smithi167.stdout:Install 6 Packages 2023-10-05T20:58:26.244 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T20:58:26.245 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 53 M 2023-10-05T20:58:26.245 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 234 M 2023-10-05T20:58:26.245 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2023-10-05T20:58:26.607 INFO:teuthology.orchestra.run.smithi167.stdout:(1/6): libcephsqlite-17.2.6-1278.g487ffcb5.el8. 543 kB/s | 174 kB 00:00 2023-10-05T20:58:28.715 INFO:teuthology.orchestra.run.smithi167.stdout:(2/6): ceph-test-17.2.6-1278.g487ffcb5.el8.x86_ 21 MB/s | 52 MB 00:02 2023-10-05T21:03:26.609 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] jq-1.6-7.el8.x86_64.rpm: Curl error (28): Timeout was reached for http://mirrors.cmich.edu/centos/8-stream/AppStream/x86_64/os/Packages/jq-1.6-7.el8.x86_64.rpm [Operation too slow. Less than 1000 bytes/sec transferred the last 300 seconds] 2023-10-05T21:03:27.232 INFO:teuthology.orchestra.run.smithi167.stdout:(3/6): jq-1.6-7.el8.x86_64.rpm 689 B/s | 202 kB 05:00 2023-10-05T21:03:27.366 INFO:teuthology.orchestra.run.smithi167.stdout:(4/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 529 kB/s | 71 kB 00:00 2023-10-05T21:03:36.618 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] socat-1.7.4.1-1.el8.x86_64.rpm: Curl error (28): Timeout was reached for http://mirrors.cmich.edu/centos/8-stream/AppStream/x86_64/os/Packages/socat-1.7.4.1-1.el8.x86_64.rpm [Operation too slow. Less than 1000 bytes/sec transferred the last 300 seconds] 2023-10-05T21:03:37.306 INFO:teuthology.orchestra.run.smithi167.stdout:(5/6): socat-1.7.4.1-1.el8.x86_64.rpm 1.0 kB/s | 323 kB 05:08 2023-10-05T21:03:42.623 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] oniguruma-6.8.2-2.el8.x86_64.rpm: Curl error (28): Timeout was reached for http://mirrors.cmich.edu/centos/8-stream/AppStream/x86_64/os/Packages/oniguruma-6.8.2-2.el8.x86_64.rpm [Operation too slow. Less than 1000 bytes/sec transferred the last 300 seconds] 2023-10-05T21:03:43.181 INFO:teuthology.orchestra.run.smithi167.stdout:(6/6): oniguruma-6.8.2-2.el8.x86_64.rpm 606 B/s | 187 kB 05:16 2023-10-05T21:03:43.181 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2023-10-05T21:03:43.181 INFO:teuthology.orchestra.run.smithi167.stdout:Total 170 kB/s | 53 MB 05:16 2023-10-05T21:03:43.222 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2023-10-05T21:03:43.244 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2023-10-05T21:03:43.244 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2023-10-05T21:03:43.561 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2023-10-05T21:03:43.564 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2023-10-05T21:03:44.039 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2023-10-05T21:03:44.263 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2023-10-05T21:03:44.460 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2023-10-05T21:03:44.599 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 3/6 2023-10-05T21:03:44.678 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 3/6 2023-10-05T21:03:44.856 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : jq-1.6-7.el8.x86_64 4/6 2023-10-05T21:03:44.999 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephsqlite-2:17.2.6-1278.g487ffcb5.el8.x86_64 5/6 2023-10-05T21:03:50.052 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1278.g487ffcb5.el8.x86_64 5/6 2023-10-05T21:03:50.263 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-test-2:17.2.6-1278.g487ffcb5.el8.x86_64 6/6 2023-10-05T21:03:50.532 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-test-2:17.2.6-1278.g487ffcb5.el8.x86_64 6/6 2023-10-05T21:03:50.532 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-test-2:17.2.6-1278.g487ffcb5.el8.x86_64 1/6 2023-10-05T21:03:50.532 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephsqlite-2:17.2.6-1278.g487ffcb5.el8.x86_64 2/6 2023-10-05T21:03:50.532 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : jq-1.6-7.el8.x86_64 3/6 2023-10-05T21:03:50.532 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 4/6 2023-10-05T21:03:50.532 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2023-10-05T21:03:51.201 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2023-10-05T21:03:51.201 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:03:51.201 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2023-10-05T21:03:51.201 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test-2:17.2.6-1278.g487ffcb5.el8.x86_64 2023-10-05T21:03:51.201 INFO:teuthology.orchestra.run.smithi167.stdout: jq-1.6-7.el8.x86_64 2023-10-05T21:03:51.201 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite-2:17.2.6-1278.g487ffcb5.el8.x86_64 2023-10-05T21:03:51.201 INFO:teuthology.orchestra.run.smithi167.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-05T21:03:51.201 INFO:teuthology.orchestra.run.smithi167.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-05T21:03:51.201 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-05T21:03:51.201 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:03:51.202 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-10-05T21:03:51.434 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph 2023-10-05T21:03:52.307 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:11:12 ago on Thu 05 Oct 2023 08:52:40 PM UTC. 2023-10-05T21:03:53.903 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-10-05T21:03:53.906 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:03:53.906 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2023-10-05T21:03:53.906 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:03:53.906 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2023-10-05T21:03:53.906 INFO:teuthology.orchestra.run.smithi167.stdout: ceph x86_64 2:17.2.6-1278.g487ffcb5.el8 ceph 6.4 k 2023-10-05T21:03:53.907 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2023-10-05T21:03:53.907 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds x86_64 2:17.2.6-1278.g487ffcb5.el8 ceph 2.3 M 2023-10-05T21:03:53.907 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr x86_64 2:17.2.6-1278.g487ffcb5.el8 ceph 1.6 M 2023-10-05T21:03:53.907 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1278.g487ffcb5.el8 ceph-noarch 238 k 2023-10-05T21:03:53.907 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon x86_64 2:17.2.6-1278.g487ffcb5.el8 ceph 3.9 M 2023-10-05T21:03:53.907 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd x86_64 2:17.2.6-1278.g487ffcb5.el8 ceph 17 M 2023-10-05T21:03:53.907 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-05T21:03:53.907 INFO:teuthology.orchestra.run.smithi167.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-05T21:03:53.907 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2023-10-05T21:03:53.907 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-05T21:03:53.907 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-05T21:03:53.907 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cryptography x86_64 3.2.1-6.el8 baseos 641 k 2023-10-05T21:03:53.907 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-05T21:03:53.908 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-05T21:03:53.908 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-05T21:03:53.908 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2023-10-05T21:03:53.908 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2023-10-05T21:03:53.908 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-05T21:03:53.908 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-05T21:03:53.908 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-05T21:03:53.908 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2023-10-05T21:03:53.908 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2023-10-05T21:03:53.908 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2023-10-05T21:03:53.908 INFO:teuthology.orchestra.run.smithi167.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-05T21:03:53.908 INFO:teuthology.orchestra.run.smithi167.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-05T21:03:53.908 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-05T21:03:53.908 INFO:teuthology.orchestra.run.smithi167.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-05T21:03:53.909 INFO:teuthology.orchestra.run.smithi167.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-05T21:03:53.909 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-05T21:03:53.909 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-05T21:03:53.909 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2023-10-05T21:03:53.909 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-05T21:03:53.909 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:03:53.909 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-10-05T21:03:53.909 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:03:53.909 INFO:teuthology.orchestra.run.smithi167.stdout:Install 32 Packages 2023-10-05T21:03:53.909 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:03:53.910 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 28 M 2023-10-05T21:03:53.910 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 110 M 2023-10-05T21:03:53.910 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2023-10-05T21:03:54.517 INFO:teuthology.orchestra.run.smithi167.stdout:(1/32): ceph-17.2.6-1278.g487ffcb5.el8.x86_64.r 35 kB/s | 6.4 kB 00:00 2023-10-05T21:03:54.642 INFO:teuthology.orchestra.run.smithi167.stdout:(2/32): ceph-mgr-17.2.6-1278.g487ffcb5.el8.x86_ 5.1 MB/s | 1.6 MB 00:00 2023-10-05T21:03:55.026 INFO:teuthology.orchestra.run.smithi167.stdout:(3/32): ceph-mds-17.2.6-1278.g487ffcb5.el8.x86_ 3.3 MB/s | 2.3 MB 00:00 2023-10-05T21:03:55.127 INFO:teuthology.orchestra.run.smithi167.stdout:(4/32): ceph-mgr-modules-core-17.2.6-1278.g487f 2.3 MB/s | 238 kB 00:00 2023-10-05T21:03:55.277 INFO:teuthology.orchestra.run.smithi167.stdout:(5/32): python3-mako-1.0.6-14.el8.noarch.rpm 1.0 MB/s | 157 kB 00:00 2023-10-05T21:03:55.336 INFO:teuthology.orchestra.run.smithi167.stdout:(6/32): python3-markupsafe-0.23-19.el8.x86_64.r 662 kB/s | 39 kB 00:00 2023-10-05T21:03:55.553 INFO:teuthology.orchestra.run.smithi167.stdout:(7/32): ceph-osd-17.2.6-1278.g487ffcb5.el8.x86_ 18 MB/s | 17 MB 00:00 2023-10-05T21:03:55.645 INFO:teuthology.orchestra.run.smithi167.stdout:(8/32): python3-pytz-2017.2-11.el8.noarch.rpm 623 kB/s | 57 kB 00:00 2023-10-05T21:03:55.728 INFO:teuthology.orchestra.run.smithi167.stdout:(9/32): ceph-mon-17.2.6-1278.g487ffcb5.el8.x86_ 3.3 MB/s | 3.9 MB 00:01 2023-10-05T21:03:55.804 INFO:teuthology.orchestra.run.smithi167.stdout:(10/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 3.2 MB/s | 249 kB 00:00 2023-10-05T21:03:55.845 INFO:teuthology.orchestra.run.smithi167.stdout:(11/32): python3-pyOpenSSL-19.0.0-1.el8.noarch. 203 kB/s | 103 kB 00:00 2023-10-05T21:03:55.871 INFO:teuthology.orchestra.run.smithi167.stdout:(12/32): python3-cryptography-3.2.1-6.el8.x86_6 9.4 MB/s | 641 kB 00:00 2023-10-05T21:03:55.929 INFO:teuthology.orchestra.run.smithi167.stdout:(13/32): python3-pycparser-2.14-14.el8.noarch.r 1.3 MB/s | 109 kB 00:00 2023-10-05T21:03:56.130 INFO:teuthology.orchestra.run.smithi167.stdout:(14/32): python3-werkzeug-0.12.2-4.el8.noarch.r 944 kB/s | 457 kB 00:00 2023-10-05T21:03:56.222 INFO:teuthology.orchestra.run.smithi167.stdout:(15/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 124 kB/s | 44 kB 00:00 2023-10-05T21:03:56.364 INFO:teuthology.orchestra.run.smithi167.stdout:(16/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 427 kB/s | 185 kB 00:00 2023-10-05T21:03:56.464 INFO:teuthology.orchestra.run.smithi167.stdout:(17/32): python3-jaraco-6.2-6.el8.noarch.rpm 111 kB/s | 11 kB 00:00 2023-10-05T21:03:56.489 INFO:teuthology.orchestra.run.smithi167.stdout:(18/32): python3-cherrypy-18.4.0-1.el8.noarch.r 1.4 MB/s | 384 kB 00:00 2023-10-05T21:03:56.564 INFO:teuthology.orchestra.run.smithi167.stdout:(19/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 397 kB/s | 173 kB 00:00 2023-10-05T21:03:56.590 INFO:teuthology.orchestra.run.smithi167.stdout:(20/32): python3-jaraco-functools-2.0-4.el8.noa 140 kB/s | 18 kB 00:00 2023-10-05T21:03:56.615 INFO:teuthology.orchestra.run.smithi167.stdout:(21/32): python3-logutils-0.3.5-11.el8.noarch.r 388 kB/s | 49 kB 00:00 2023-10-05T21:03:56.674 INFO:teuthology.orchestra.run.smithi167.stdout:(22/32): python3-more-itertools-7.2.0-3.el8.noa 539 kB/s | 59 kB 00:00 2023-10-05T21:03:56.715 INFO:teuthology.orchestra.run.smithi167.stdout:(23/32): python3-portend-2.6-1.el8.noarch.rpm 155 kB/s | 16 kB 00:00 2023-10-05T21:03:56.774 INFO:teuthology.orchestra.run.smithi167.stdout:(24/32): python3-pecan-1.3.2-9.el8.noarch.rpm 1.5 MB/s | 283 kB 00:00 2023-10-05T21:03:56.799 INFO:teuthology.orchestra.run.smithi167.stdout:(25/32): python3-simplegeneric-0.8.1-17.el8.noa 149 kB/s | 19 kB 00:00 2023-10-05T21:03:56.833 INFO:teuthology.orchestra.run.smithi167.stdout:(26/32): python3-singledispatch-3.4.0.3-18.el8. 219 kB/s | 24 kB 00:00 2023-10-05T21:03:56.875 INFO:teuthology.orchestra.run.smithi167.stdout:(27/32): python3-tempora-1.14.1-5.el8.noarch.rp 291 kB/s | 29 kB 00:00 2023-10-05T21:03:56.900 INFO:teuthology.orchestra.run.smithi167.stdout:(28/32): python3-trustme-0.6.0-4.el8.noarch.rpm 274 kB/s | 27 kB 00:00 2023-10-05T21:03:56.934 INFO:teuthology.orchestra.run.smithi167.stdout:(29/32): python3-waitress-1.4.3-1.el8.noarch.rp 2.3 MB/s | 255 kB 00:00 2023-10-05T21:03:56.984 INFO:teuthology.orchestra.run.smithi167.stdout:(30/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 2.2 MB/s | 251 kB 00:00 2023-10-05T21:03:57.010 INFO:teuthology.orchestra.run.smithi167.stdout:(31/32): python3-webtest-2.0.33-1.el8.noarch.rp 785 kB/s | 85 kB 00:00 2023-10-05T21:03:57.035 INFO:teuthology.orchestra.run.smithi167.stdout:(32/32): python3-zc-lockfile-2.0-2.el8.noarch.r 229 kB/s | 23 kB 00:00 2023-10-05T21:03:57.036 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2023-10-05T21:03:57.037 INFO:teuthology.orchestra.run.smithi167.stdout:Total 9.1 MB/s | 28 MB 00:03 2023-10-05T21:03:57.121 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2023-10-05T21:03:57.158 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2023-10-05T21:03:57.158 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2023-10-05T21:03:57.462 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2023-10-05T21:03:57.466 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2023-10-05T21:03:58.529 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2023-10-05T21:03:58.728 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2023-10-05T21:03:58.955 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2023-10-05T21:03:59.172 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2023-10-05T21:03:59.386 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2023-10-05T21:03:59.584 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2023-10-05T21:03:59.782 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2023-10-05T21:03:59.967 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2023-10-05T21:04:00.169 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2023-10-05T21:04:00.362 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2023-10-05T21:04:00.591 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2023-10-05T21:04:00.812 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2023-10-05T21:04:01.061 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2023-10-05T21:04:01.466 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2023-10-05T21:04:01.749 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 14/32 2023-10-05T21:04:01.939 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2023-10-05T21:04:02.162 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2023-10-05T21:04:02.372 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-10-05T21:04:02.690 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2023-10-05T21:04:02.885 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2023-10-05T21:04:03.115 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/32 2023-10-05T21:04:03.301 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2023-10-05T21:04:03.572 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2023-10-05T21:04:03.826 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2023-10-05T21:04:04.049 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2023-10-05T21:04:04.663 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2023-10-05T21:04:04.977 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2023-10-05T21:04:05.323 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1278.g487ffcb5.el8. 27/32 2023-10-05T21:04:05.501 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-2:17.2.6-1278.g487ffcb5.el8.x86_64 28/32 2023-10-05T21:04:06.979 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1278.g487ffcb5.el8.x86_64 28/32 2023-10-05T21:04:07.143 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-osd-2:17.2.6-1278.g487ffcb5.el8.x86_64 29/32 2023-10-05T21:04:07.537 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-osd-2:17.2.6-1278.g487ffcb5.el8.x86_64 29/32 2023-10-05T21:04:07.712 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mon-2:17.2.6-1278.g487ffcb5.el8.x86_64 30/32 2023-10-05T21:04:07.930 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mon-2:17.2.6-1278.g487ffcb5.el8.x86_64 30/32 2023-10-05T21:04:08.079 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mds-2:17.2.6-1278.g487ffcb5.el8.x86_64 31/32 2023-10-05T21:04:08.097 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mds-2:17.2.6-1278.g487ffcb5.el8.x86_64 31/32 2023-10-05T21:04:08.212 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-2:17.2.6-1278.g487ffcb5.el8.x86_64 32/32 2023-10-05T21:04:08.604 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-2:17.2.6-1278.g487ffcb5.el8.x86_64 32/32 2023-10-05T21:04:08.604 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-2:17.2.6-1278.g487ffcb5.el8.x86_64 1/32 2023-10-05T21:04:08.605 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mds-2:17.2.6-1278.g487ffcb5.el8.x86_64 2/32 2023-10-05T21:04:08.605 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-2:17.2.6-1278.g487ffcb5.el8.x86_64 3/32 2023-10-05T21:04:08.605 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mon-2:17.2.6-1278.g487ffcb5.el8.x86_64 4/32 2023-10-05T21:04:08.605 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-osd-2:17.2.6-1278.g487ffcb5.el8.x86_64 5/32 2023-10-05T21:04:08.605 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1278.g487ffcb5.el8. 6/32 2023-10-05T21:04:08.605 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2023-10-05T21:04:08.605 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2023-10-05T21:04:08.605 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2023-10-05T21:04:08.605 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 10/32 2023-10-05T21:04:08.605 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2023-10-05T21:04:08.606 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2023-10-05T21:04:08.606 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 13/32 2023-10-05T21:04:08.606 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2023-10-05T21:04:08.606 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2023-10-05T21:04:08.606 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2023-10-05T21:04:08.606 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-10-05T21:04:08.606 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2023-10-05T21:04:08.606 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2023-10-05T21:04:08.606 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2023-10-05T21:04:08.606 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2023-10-05T21:04:08.606 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2023-10-05T21:04:08.606 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2023-10-05T21:04:08.607 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2023-10-05T21:04:08.607 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2023-10-05T21:04:08.607 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2023-10-05T21:04:08.607 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2023-10-05T21:04:08.607 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2023-10-05T21:04:08.607 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2023-10-05T21:04:08.607 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2023-10-05T21:04:08.607 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2023-10-05T21:04:09.720 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2023-10-05T21:04:09.720 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:04:09.720 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2023-10-05T21:04:09.720 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-2:17.2.6-1278.g487ffcb5.el8.x86_64 2023-10-05T21:04:09.720 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds-2:17.2.6-1278.g487ffcb5.el8.x86_64 2023-10-05T21:04:09.720 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-2:17.2.6-1278.g487ffcb5.el8.x86_64 2023-10-05T21:04:09.720 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core-2:17.2.6-1278.g487ffcb5.el8.noarch 2023-10-05T21:04:09.721 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon-2:17.2.6-1278.g487ffcb5.el8.x86_64 2023-10-05T21:04:09.721 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd-2:17.2.6-1278.g487ffcb5.el8.x86_64 2023-10-05T21:04:09.721 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-05T21:04:09.721 INFO:teuthology.orchestra.run.smithi167.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-05T21:04:09.721 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-10-05T21:04:09.721 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-05T21:04:09.721 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-05T21:04:09.721 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-10-05T21:04:09.721 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-05T21:04:09.721 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-05T21:04:09.721 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-05T21:04:09.721 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako-1.0.6-14.el8.noarch 2023-10-05T21:04:09.722 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-05T21:04:09.722 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-05T21:04:09.722 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-05T21:04:09.722 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend-2.6-1.el8.noarch 2023-10-05T21:04:09.722 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-05T21:04:09.722 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-05T21:04:09.722 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz-2017.2-11.el8.noarch 2023-10-05T21:04:09.722 INFO:teuthology.orchestra.run.smithi167.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-05T21:04:09.722 INFO:teuthology.orchestra.run.smithi167.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-05T21:04:09.722 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-05T21:04:09.722 INFO:teuthology.orchestra.run.smithi167.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-05T21:04:09.722 INFO:teuthology.orchestra.run.smithi167.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-05T21:04:09.723 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-05T21:04:09.723 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-05T21:04:09.723 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-05T21:04:09.723 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-05T21:04:09.723 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:04:09.723 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-10-05T21:04:10.007 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-base 2023-10-05T21:04:10.867 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:11:30 ago on Thu 05 Oct 2023 08:52:40 PM UTC. 2023-10-05T21:04:12.260 INFO:teuthology.orchestra.run.smithi167.stdout:Package ceph-base-2:17.2.6-1278.g487ffcb5.el8.x86_64 is already installed. 2023-10-05T21:04:12.446 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-10-05T21:04:12.447 INFO:teuthology.orchestra.run.smithi167.stdout:Nothing to do. 2023-10-05T21:04:12.447 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-10-05T21:04:12.557 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install cephadm 2023-10-05T21:04:13.430 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:11:33 ago on Thu 05 Oct 2023 08:52:40 PM UTC. 2023-10-05T21:04:15.009 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-10-05T21:04:15.010 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:04:15.010 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2023-10-05T21:04:15.010 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:04:15.011 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2023-10-05T21:04:15.011 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm noarch 2:17.2.6-1278.g487ffcb5.el8 ceph-noarch 80 k 2023-10-05T21:04:15.011 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:04:15.011 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-10-05T21:04:15.011 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:04:15.011 INFO:teuthology.orchestra.run.smithi167.stdout:Install 1 Package 2023-10-05T21:04:15.011 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:04:15.011 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 80 k 2023-10-05T21:04:15.011 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 364 k 2023-10-05T21:04:15.011 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2023-10-05T21:04:15.177 INFO:teuthology.orchestra.run.smithi167.stdout:cephadm-17.2.6-1278.g487ffcb5.el8.noarch.rpm 484 kB/s | 80 kB 00:00 2023-10-05T21:04:15.178 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2023-10-05T21:04:15.178 INFO:teuthology.orchestra.run.smithi167.stdout:Total 481 kB/s | 80 kB 00:00 2023-10-05T21:04:15.178 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2023-10-05T21:04:15.186 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2023-10-05T21:04:15.186 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2023-10-05T21:04:15.188 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2023-10-05T21:04:15.191 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2023-10-05T21:04:15.236 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2023-10-05T21:04:15.740 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephadm-2:17.2.6-1278.g487ffcb5.el8.noarch 1/1 2023-10-05T21:04:15.896 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephadm-2:17.2.6-1278.g487ffcb5.el8.noarch 1/1 2023-10-05T21:04:16.121 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephadm-2:17.2.6-1278.g487ffcb5.el8.noarch 1/1 2023-10-05T21:04:16.721 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephadm-2:17.2.6-1278.g487ffcb5.el8.noarch 1/1 2023-10-05T21:04:16.721 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:04:16.721 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2023-10-05T21:04:16.721 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm-2:17.2.6-1278.g487ffcb5.el8.noarch 2023-10-05T21:04:16.722 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:04:16.722 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-10-05T21:04:16.960 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-immutable-object-cache 2023-10-05T21:04:17.817 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:11:37 ago on Thu 05 Oct 2023 08:52:40 PM UTC. 2023-10-05T21:04:19.396 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-10-05T21:04:19.397 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:04:19.397 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repo Size 2023-10-05T21:04:19.397 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:04:19.397 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2023-10-05T21:04:19.397 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1278.g487ffcb5.el8 ceph 162 k 2023-10-05T21:04:19.398 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:04:19.398 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-10-05T21:04:19.398 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:04:19.398 INFO:teuthology.orchestra.run.smithi167.stdout:Install 1 Package 2023-10-05T21:04:19.398 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:04:19.399 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 162 k 2023-10-05T21:04:19.399 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 490 k 2023-10-05T21:04:19.399 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2023-10-05T21:04:19.699 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-immutable-object-cache-17.2.6-1278.g487ffc 537 kB/s | 162 kB 00:00 2023-10-05T21:04:19.700 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2023-10-05T21:04:19.700 INFO:teuthology.orchestra.run.smithi167.stdout:Total 535 kB/s | 162 kB 00:00 2023-10-05T21:04:19.700 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2023-10-05T21:04:19.710 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2023-10-05T21:04:19.711 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2023-10-05T21:04:19.756 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2023-10-05T21:04:19.760 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2023-10-05T21:04:19.866 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2023-10-05T21:04:20.824 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1278.g487ffcb5. 1/1 2023-10-05T21:04:22.022 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1278.g487ffcb5. 1/1 2023-10-05T21:04:22.690 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1278.g487ffcb5. 1/1 2023-10-05T21:04:22.690 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:04:22.691 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2023-10-05T21:04:22.691 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache-2:17.2.6-1278.g487ffcb5.el8.x86_64 2023-10-05T21:04:22.691 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:04:22.691 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-10-05T21:04:22.905 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-mgr 2023-10-05T21:04:23.783 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:11:43 ago on Thu 05 Oct 2023 08:52:40 PM UTC. 2023-10-05T21:04:25.186 INFO:teuthology.orchestra.run.smithi167.stdout:Package ceph-mgr-2:17.2.6-1278.g487ffcb5.el8.x86_64 is already installed. 2023-10-05T21:04:25.379 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-10-05T21:04:25.380 INFO:teuthology.orchestra.run.smithi167.stdout:Nothing to do. 2023-10-05T21:04:25.381 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-10-05T21:04:25.488 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-mgr-dashboard 2023-10-05T21:04:26.347 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:11:46 ago on Thu 05 Oct 2023 08:52:40 PM UTC. 2023-10-05T21:04:27.922 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-10-05T21:04:27.923 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:04:27.924 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2023-10-05T21:04:27.924 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:04:27.924 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2023-10-05T21:04:27.924 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1278.g487ffcb5.el8 ceph-noarch 1.7 M 2023-10-05T21:04:27.924 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2023-10-05T21:04:27.924 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1278.g487ffcb5.el8 ceph-noarch 23 k 2023-10-05T21:04:27.924 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1278.g487ffcb5.el8 ceph-noarch 14 k 2023-10-05T21:04:27.924 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2023-10-05T21:04:27.924 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-05T21:04:27.924 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-05T21:04:27.925 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:04:27.925 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-10-05T21:04:27.925 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:04:27.925 INFO:teuthology.orchestra.run.smithi167.stdout:Install 6 Packages 2023-10-05T21:04:27.925 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:04:27.925 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 2.0 M 2023-10-05T21:04:27.926 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 8.0 M 2023-10-05T21:04:27.926 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2023-10-05T21:04:28.345 INFO:teuthology.orchestra.run.smithi167.stdout:(1/6): ceph-grafana-dashboards-17.2.6-1278.g487 149 kB/s | 23 kB 00:00 2023-10-05T21:04:28.404 INFO:teuthology.orchestra.run.smithi167.stdout:(2/6): python3-jwt-1.6.1-2.el8.noarch.rpm 732 kB/s | 43 kB 00:00 2023-10-05T21:04:28.438 INFO:teuthology.orchestra.run.smithi167.stdout:(3/6): ceph-prometheus-alerts-17.2.6-1278.g487f 59 kB/s | 14 kB 00:00 2023-10-05T21:04:28.555 INFO:teuthology.orchestra.run.smithi167.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 223 kB/s | 34 kB 00:00 2023-10-05T21:04:28.613 INFO:teuthology.orchestra.run.smithi167.stdout:(5/6): python3-routes-2.4.1-12.el8.noarch.rpm 1.1 MB/s | 196 kB 00:00 2023-10-05T21:04:28.689 INFO:teuthology.orchestra.run.smithi167.stdout:(6/6): ceph-mgr-dashboard-17.2.6-1278.g487ffcb5 3.4 MB/s | 1.7 MB 00:00 2023-10-05T21:04:28.692 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2023-10-05T21:04:28.692 INFO:teuthology.orchestra.run.smithi167.stdout:Total 2.6 MB/s | 2.0 MB 00:00 2023-10-05T21:04:28.703 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2023-10-05T21:04:28.711 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2023-10-05T21:04:28.712 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2023-10-05T21:04:28.791 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2023-10-05T21:04:28.794 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2023-10-05T21:04:29.041 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2023-10-05T21:04:29.224 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2023-10-05T21:04:29.460 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2023-10-05T21:04:29.645 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2023-10-05T21:04:29.765 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1278.g487ffcb5.el8.n 4/6 2023-10-05T21:04:30.105 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1278.g487ffcb5.el8. 5/6 2023-10-05T21:04:30.314 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1278.g487ffcb5.el8.noarc 6/6 2023-10-05T21:04:30.599 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1278.g487ffcb5.el8.noarc 6/6 2023-10-05T21:04:30.599 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1278.g487ffcb5.el8. 1/6 2023-10-05T21:04:30.599 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1278.g487ffcb5.el8.noarc 2/6 2023-10-05T21:04:30.599 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1278.g487ffcb5.el8.n 3/6 2023-10-05T21:04:30.599 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2023-10-05T21:04:30.599 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2023-10-05T21:04:31.229 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2023-10-05T21:04:31.229 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:04:31.229 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2023-10-05T21:04:31.229 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards-2:17.2.6-1278.g487ffcb5.el8.noarch 2023-10-05T21:04:31.229 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard-2:17.2.6-1278.g487ffcb5.el8.noarch 2023-10-05T21:04:31.229 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts-2:17.2.6-1278.g487ffcb5.el8.noarch 2023-10-05T21:04:31.230 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-05T21:04:31.230 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-05T21:04:31.230 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-05T21:04:31.230 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:04:31.230 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-10-05T21:04:31.354 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-mgr-diskprediction-local 2023-10-05T21:04:32.208 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:11:52 ago on Thu 05 Oct 2023 08:52:40 PM UTC. 2023-10-05T21:04:33.802 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-10-05T21:04:33.803 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================================ 2023-10-05T21:04:33.803 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2023-10-05T21:04:33.803 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================================ 2023-10-05T21:04:33.803 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2023-10-05T21:04:33.803 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1278.g487ffcb5.el8 ceph-noarch 7.4 M 2023-10-05T21:04:33.803 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2023-10-05T21:04:33.803 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran x86_64 8.5.0-20.el8 baseos 645 k 2023-10-05T21:04:33.804 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath x86_64 8.5.0-20.el8 baseos 172 k 2023-10-05T21:04:33.804 INFO:teuthology.orchestra.run.smithi167.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2023-10-05T21:04:33.804 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2023-10-05T21:04:33.804 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2023-10-05T21:04:33.804 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2023-10-05T21:04:33.804 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 appstream 14 M 2023-10-05T21:04:33.804 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:04:33.804 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-10-05T21:04:33.804 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================================ 2023-10-05T21:04:33.804 INFO:teuthology.orchestra.run.smithi167.stdout:Install 8 Packages 2023-10-05T21:04:33.804 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:04:33.805 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 35 M 2023-10-05T21:04:33.805 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 210 M 2023-10-05T21:04:33.805 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2023-10-05T21:04:34.762 INFO:teuthology.orchestra.run.smithi167.stdout:(1/8): ceph-mgr-diskprediction-local-17.2.6-127 8.5 MB/s | 7.4 MB 00:00 2023-10-05T21:04:36.871 INFO:teuthology.orchestra.run.smithi167.stdout:(2/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 1.6 MB/s | 4.9 MB 00:02 2023-10-05T21:04:36.963 INFO:teuthology.orchestra.run.smithi167.stdout:(3/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 2.4 MB/s | 225 kB 00:00 2023-10-05T21:04:38.473 INFO:teuthology.orchestra.run.smithi167.stdout:(4/8): openblas-0.3.15-4.el8.x86_64.rpm 1.0 MB/s | 4.7 MB 00:04 2023-10-05T21:04:38.573 INFO:teuthology.orchestra.run.smithi167.stdout:(5/8): libgfortran-8.5.0-20.el8.x86_64.rpm 6.3 MB/s | 645 kB 00:00 2023-10-05T21:04:38.606 INFO:teuthology.orchestra.run.smithi167.stdout:(6/8): libquadmath-8.5.0-20.el8.x86_64.rpm 5.1 MB/s | 172 kB 00:00 2023-10-05T21:04:38.674 INFO:teuthology.orchestra.run.smithi167.stdout:(7/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 977 kB/s | 3.7 MB 00:03 2023-10-05T21:04:44.824 INFO:teuthology.orchestra.run.smithi167.stdout:(8/8): python3-scipy-1.0.0-21.module_el8.5.0+77 1.7 MB/s | 14 MB 00:07 2023-10-05T21:04:44.824 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2023-10-05T21:04:44.825 INFO:teuthology.orchestra.run.smithi167.stdout:Total 3.2 MB/s | 35 MB 00:11 2023-10-05T21:04:45.061 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2023-10-05T21:04:45.078 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2023-10-05T21:04:45.079 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2023-10-05T21:04:45.469 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2023-10-05T21:04:45.472 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2023-10-05T21:04:46.028 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2023-10-05T21:04:46.165 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 1/8 2023-10-05T21:04:46.288 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 1/8 2023-10-05T21:04:46.414 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 2/8 2023-10-05T21:04:47.012 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 2/8 2023-10-05T21:04:47.206 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2023-10-05T21:04:47.824 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2023-10-05T21:04:47.973 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-10-05T21:04:48.697 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-10-05T21:04:49.052 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-10-05T21:04:49.289 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-10-05T21:04:51.034 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-10-05T21:04:52.278 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 7/8 2023-10-05T21:04:52.506 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1278.g487ffcb 8/8 2023-10-05T21:04:52.828 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1278.g487ffcb 8/8 2023-10-05T21:04:52.828 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1278.g487ffcb 1/8 2023-10-05T21:04:52.828 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2023-10-05T21:04:52.828 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2023-10-05T21:04:52.828 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2023-10-05T21:04:52.828 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2023-10-05T21:04:52.828 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 6/8 2023-10-05T21:04:52.828 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 7/8 2023-10-05T21:04:53.479 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 8/8 2023-10-05T21:04:53.479 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:04:53.479 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2023-10-05T21:04:53.479 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1278.g487ffcb5.el8.noarch 2023-10-05T21:04:53.479 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran-8.5.0-20.el8.x86_64 2023-10-05T21:04:53.479 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath-8.5.0-20.el8.x86_64 2023-10-05T21:04:53.479 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-0.3.15-4.el8.x86_64 2023-10-05T21:04:53.480 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-10-05T21:04:53.480 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-05T21:04:53.480 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-05T21:04:53.480 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-10-05T21:04:53.480 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:04:53.480 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-10-05T21:04:53.609 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-mgr-rook 2023-10-05T21:04:54.471 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:12:14 ago on Thu 05 Oct 2023 08:52:40 PM UTC. 2023-10-05T21:04:56.032 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-10-05T21:04:56.034 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:04:56.034 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2023-10-05T21:04:56.034 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:04:56.034 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2023-10-05T21:04:56.034 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook noarch 2:17.2.6-1278.g487ffcb5.el8 ceph-noarch 47 k 2023-10-05T21:04:56.035 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2023-10-05T21:04:56.035 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-05T21:04:56.035 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-05T21:04:56.035 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-05T21:04:56.035 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2023-10-05T21:04:56.035 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2023-10-05T21:04:56.035 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-05T21:04:56.035 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2023-10-05T21:04:56.035 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2023-10-05T21:04:56.035 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2023-10-05T21:04:56.035 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2023-10-05T21:04:56.036 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-05T21:04:56.036 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-05T21:04:56.036 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:04:56.036 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-10-05T21:04:56.036 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:04:56.036 INFO:teuthology.orchestra.run.smithi167.stdout:Install 13 Packages 2023-10-05T21:04:56.036 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:04:56.037 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 1.9 M 2023-10-05T21:04:56.037 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 25 M 2023-10-05T21:04:56.037 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2023-10-05T21:04:56.825 INFO:teuthology.orchestra.run.smithi167.stdout:(1/13): python3-jsonpointer-1.10-11.el8.noarch. 283 kB/s | 20 kB 00:00 2023-10-05T21:04:56.849 INFO:teuthology.orchestra.run.smithi167.stdout:(2/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 277 kB/s | 27 kB 00:00 2023-10-05T21:04:56.933 INFO:teuthology.orchestra.run.smithi167.stdout:(3/13): ceph-mgr-rook-17.2.6-1278.g487ffcb5.el8 264 kB/s | 47 kB 00:00 2023-10-05T21:04:57.000 INFO:teuthology.orchestra.run.smithi167.stdout:(4/13): python3-pyasn1-modules-0.3.7-6.el8.noar 732 kB/s | 110 kB 00:00 2023-10-05T21:04:57.025 INFO:teuthology.orchestra.run.smithi167.stdout:(5/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 631 kB/s | 126 kB 00:00 2023-10-05T21:04:57.050 INFO:teuthology.orchestra.run.smithi167.stdout:(6/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 1.3 MB/s | 155 kB 00:00 2023-10-05T21:04:57.067 INFO:teuthology.orchestra.run.smithi167.stdout:(7/13): python3-requests-oauthlib-1.0.0-1.el8.n 638 kB/s | 43 kB 00:00 2023-10-05T21:04:57.334 INFO:teuthology.orchestra.run.smithi167.stdout:(8/13): python3-certifi-2018.10.15-7.el8.noarch 57 kB/s | 16 kB 00:00 2023-10-05T21:04:57.359 INFO:teuthology.orchestra.run.smithi167.stdout:(9/13): python3-cachetools-3.1.1-4.el8.noarch.r 107 kB/s | 33 kB 00:00 2023-10-05T21:04:57.409 INFO:teuthology.orchestra.run.smithi167.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 237 kB/s | 81 kB 00:00 2023-10-05T21:04:57.485 INFO:teuthology.orchestra.run.smithi167.stdout:(11/13): python3-rsa-4.9-2.el8.noarch.rpm 513 kB/s | 64 kB 00:00 2023-10-05T21:04:57.501 INFO:teuthology.orchestra.run.smithi167.stdout:(12/13): python3-websocket-client-0.56.0-5.el8. 667 kB/s | 61 kB 00:00 2023-10-05T21:04:57.677 INFO:teuthology.orchestra.run.smithi167.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 3.4 MB/s | 1.2 MB 00:00 2023-10-05T21:04:57.679 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2023-10-05T21:04:57.680 INFO:teuthology.orchestra.run.smithi167.stdout:Total 1.2 MB/s | 1.9 MB 00:01 2023-10-05T21:04:57.741 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2023-10-05T21:04:57.757 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2023-10-05T21:04:57.758 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2023-10-05T21:04:57.845 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2023-10-05T21:04:57.848 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2023-10-05T21:04:58.333 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2023-10-05T21:04:58.592 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-10-05T21:04:58.780 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2023-10-05T21:04:58.999 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rsa-4.9-2.el8.noarch 3/13 2023-10-05T21:04:59.253 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2023-10-05T21:04:59.452 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2023-10-05T21:04:59.683 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2023-10-05T21:04:59.943 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2023-10-05T21:05:00.193 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2023-10-05T21:05:01.263 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2023-10-05T21:05:01.561 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2023-10-05T21:05:01.770 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2023-10-05T21:05:01.997 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2023-10-05T21:05:02.154 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-rook-2:17.2.6-1278.g487ffcb5.el8.noarch 13/13 2023-10-05T21:05:02.445 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1278.g487ffcb5.el8.noarch 13/13 2023-10-05T21:05:02.445 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1278.g487ffcb5.el8.noarch 1/13 2023-10-05T21:05:02.445 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2023-10-05T21:05:02.445 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2023-10-05T21:05:02.446 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2023-10-05T21:05:02.446 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2023-10-05T21:05:02.446 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2023-10-05T21:05:02.446 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2023-10-05T21:05:02.446 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2023-10-05T21:05:02.446 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-10-05T21:05:02.446 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2023-10-05T21:05:02.446 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2023-10-05T21:05:02.446 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 12/13 2023-10-05T21:05:03.129 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2023-10-05T21:05:03.129 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:05:03.129 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2023-10-05T21:05:03.130 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook-2:17.2.6-1278.g487ffcb5.el8.noarch 2023-10-05T21:05:03.130 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-05T21:05:03.130 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-05T21:05:03.130 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-05T21:05:03.130 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-05T21:05:03.130 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-05T21:05:03.130 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-05T21:05:03.130 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-05T21:05:03.130 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-05T21:05:03.130 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-05T21:05:03.130 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-05T21:05:03.130 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-05T21:05:03.131 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-05T21:05:03.131 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:05:03.131 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-10-05T21:05:03.249 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-mgr-cephadm 2023-10-05T21:05:04.108 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:12:24 ago on Thu 05 Oct 2023 08:52:40 PM UTC. 2023-10-05T21:05:05.693 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-10-05T21:05:05.694 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:05:05.694 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2023-10-05T21:05:05.694 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:05:05.694 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2023-10-05T21:05:05.694 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1278.g487ffcb5.el8 ceph-noarch 121 k 2023-10-05T21:05:05.694 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2023-10-05T21:05:05.694 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-05T21:05:05.694 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2023-10-05T21:05:05.694 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2 noarch 2.10.1-3.el8 appstream 538 k 2023-10-05T21:05:05.695 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-05T21:05:05.695 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:05:05.695 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-10-05T21:05:05.695 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:05:05.695 INFO:teuthology.orchestra.run.smithi167.stdout:Install 5 Packages 2023-10-05T21:05:05.695 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:05:05.695 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 6.0 M 2023-10-05T21:05:05.696 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 26 M 2023-10-05T21:05:05.696 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2023-10-05T21:05:06.512 INFO:teuthology.orchestra.run.smithi167.stdout:(1/5): ceph-mgr-cephadm-17.2.6-1278.g487ffcb5.e 417 kB/s | 121 kB 00:00 2023-10-05T21:05:06.737 INFO:teuthology.orchestra.run.smithi167.stdout:(2/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 1.0 MB/s | 538 kB 00:00 2023-10-05T21:05:06.865 INFO:teuthology.orchestra.run.smithi167.stdout:(3/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 1.3 MB/s | 476 kB 00:00 2023-10-05T21:05:06.946 INFO:teuthology.orchestra.run.smithi167.stdout:(4/5): python3-natsort-7.1.1-2.el8.noarch.rpm 288 kB/s | 60 kB 00:00 2023-10-05T21:05:09.174 INFO:teuthology.orchestra.run.smithi167.stdout:(5/5): python3-babel-2.5.1-7.el8.noarch.rpm 1.6 MB/s | 4.8 MB 00:02 2023-10-05T21:05:09.175 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2023-10-05T21:05:09.176 INFO:teuthology.orchestra.run.smithi167.stdout:Total 1.7 MB/s | 6.0 MB 00:03 2023-10-05T21:05:09.232 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2023-10-05T21:05:09.243 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2023-10-05T21:05:09.244 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2023-10-05T21:05:09.349 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2023-10-05T21:05:09.352 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2023-10-05T21:05:09.784 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2023-10-05T21:05:10.125 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2023-10-05T21:05:11.074 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-10-05T21:05:11.432 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2023-10-05T21:05:11.688 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 4/5 2023-10-05T21:05:11.853 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1278.g487ffcb5.el8.noarch 5/5 2023-10-05T21:05:12.143 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1278.g487ffcb5.el8.noarch 5/5 2023-10-05T21:05:12.143 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1278.g487ffcb5.el8.noarch 1/5 2023-10-05T21:05:12.143 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2023-10-05T21:05:12.143 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 3/5 2023-10-05T21:05:12.143 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-10-05T21:05:12.768 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2023-10-05T21:05:12.768 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:05:12.768 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2023-10-05T21:05:12.768 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm-2:17.2.6-1278.g487ffcb5.el8.noarch 2023-10-05T21:05:12.768 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-05T21:05:12.768 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-05T21:05:12.768 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-05T21:05:12.768 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-05T21:05:12.769 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:05:12.769 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-10-05T21:05:12.880 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-fuse 2023-10-05T21:05:13.751 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:12:33 ago on Thu 05 Oct 2023 08:52:40 PM UTC. 2023-10-05T21:05:15.339 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-10-05T21:05:15.339 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:05:15.339 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2023-10-05T21:05:15.339 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:05:15.339 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2023-10-05T21:05:15.339 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse x86_64 2:17.2.6-1278.g487ffcb5.el8 ceph 876 k 2023-10-05T21:05:15.340 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2023-10-05T21:05:15.340 INFO:teuthology.orchestra.run.smithi167.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2023-10-05T21:05:15.340 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:05:15.340 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-10-05T21:05:15.340 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:05:15.340 INFO:teuthology.orchestra.run.smithi167.stdout:Install 2 Packages 2023-10-05T21:05:15.340 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:05:15.341 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 959 k 2023-10-05T21:05:15.341 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 3.0 M 2023-10-05T21:05:15.341 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2023-10-05T21:05:15.454 INFO:teuthology.orchestra.run.smithi167.stdout:(1/2): fuse-2.9.7-17.el8.x86_64.rpm 1.1 MB/s | 83 kB 00:00 2023-10-05T21:05:15.771 INFO:teuthology.orchestra.run.smithi167.stdout:(2/2): ceph-fuse-17.2.6-1278.g487ffcb5.el8.x86_ 2.2 MB/s | 876 kB 00:00 2023-10-05T21:05:15.771 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2023-10-05T21:05:15.772 INFO:teuthology.orchestra.run.smithi167.stdout:Total 2.2 MB/s | 959 kB 00:00 2023-10-05T21:05:15.782 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2023-10-05T21:05:15.802 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2023-10-05T21:05:15.803 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2023-10-05T21:05:15.890 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2023-10-05T21:05:15.893 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2023-10-05T21:05:16.063 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2023-10-05T21:05:16.320 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : fuse-2.9.7-17.el8.x86_64 1/2 2023-10-05T21:05:16.527 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-fuse-2:17.2.6-1278.g487ffcb5.el8.x86_64 2/2 2023-10-05T21:05:16.929 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-fuse-2:17.2.6-1278.g487ffcb5.el8.x86_64 2/2 2023-10-05T21:05:16.929 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-fuse-2:17.2.6-1278.g487ffcb5.el8.x86_64 1/2 2023-10-05T21:05:17.595 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 2/2 2023-10-05T21:05:17.595 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:05:17.596 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2023-10-05T21:05:17.596 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse-2:17.2.6-1278.g487ffcb5.el8.x86_64 fuse-2.9.7-17.el8.x86_64 2023-10-05T21:05:17.596 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:05:17.596 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-10-05T21:05:17.812 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-volume 2023-10-05T21:05:18.674 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:12:38 ago on Thu 05 Oct 2023 08:52:40 PM UTC. 2023-10-05T21:05:20.254 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-10-05T21:05:20.255 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:05:20.255 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2023-10-05T21:05:20.255 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:05:20.255 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2023-10-05T21:05:20.255 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume noarch 2:17.2.6-1278.g487ffcb5.el8 ceph-noarch 267 k 2023-10-05T21:05:20.255 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:05:20.255 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-10-05T21:05:20.256 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:05:20.256 INFO:teuthology.orchestra.run.smithi167.stdout:Install 1 Package 2023-10-05T21:05:20.256 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:05:20.256 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 267 k 2023-10-05T21:05:20.256 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 1.3 M 2023-10-05T21:05:20.256 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2023-10-05T21:05:20.544 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-volume-17.2.6-1278.g487ffcb5.el8.noarch.rp 925 kB/s | 267 kB 00:00 2023-10-05T21:05:20.545 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2023-10-05T21:05:20.545 INFO:teuthology.orchestra.run.smithi167.stdout:Total 921 kB/s | 267 kB 00:00 2023-10-05T21:05:20.545 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2023-10-05T21:05:20.555 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2023-10-05T21:05:20.555 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2023-10-05T21:05:20.594 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2023-10-05T21:05:20.595 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2023-10-05T21:05:20.781 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2023-10-05T21:05:21.035 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-volume-2:17.2.6-1278.g487ffcb5.el8.noarch 1/1 2023-10-05T21:05:21.450 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-volume-2:17.2.6-1278.g487ffcb5.el8.noarch 1/1 2023-10-05T21:05:22.090 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-volume-2:17.2.6-1278.g487ffcb5.el8.noarch 1/1 2023-10-05T21:05:22.091 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:05:22.091 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2023-10-05T21:05:22.091 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume-2:17.2.6-1278.g487ffcb5.el8.noarch 2023-10-05T21:05:22.091 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:05:22.091 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-10-05T21:05:22.274 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install librados-devel 2023-10-05T21:05:23.137 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:12:43 ago on Thu 05 Oct 2023 08:52:40 PM UTC. 2023-10-05T21:05:24.723 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-10-05T21:05:24.723 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:05:24.723 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repo Size 2023-10-05T21:05:24.724 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:05:24.724 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2023-10-05T21:05:24.724 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel x86_64 2:17.2.6-1278.g487ffcb5.el8 ceph 130 k 2023-10-05T21:05:24.724 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:05:24.724 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-10-05T21:05:24.724 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:05:24.724 INFO:teuthology.orchestra.run.smithi167.stdout:Install 1 Package 2023-10-05T21:05:24.724 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:05:24.725 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 130 k 2023-10-05T21:05:24.725 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 464 k 2023-10-05T21:05:24.725 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2023-10-05T21:05:25.015 INFO:teuthology.orchestra.run.smithi167.stdout:librados-devel-17.2.6-1278.g487ffcb5.el8.x86_64 447 kB/s | 130 kB 00:00 2023-10-05T21:05:25.016 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2023-10-05T21:05:25.016 INFO:teuthology.orchestra.run.smithi167.stdout:Total 445 kB/s | 130 kB 00:00 2023-10-05T21:05:25.016 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2023-10-05T21:05:25.021 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2023-10-05T21:05:25.022 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2023-10-05T21:05:25.072 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2023-10-05T21:05:25.075 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2023-10-05T21:05:25.186 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2023-10-05T21:05:25.450 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librados-devel-2:17.2.6-1278.g487ffcb5.el8.x86_64 1/1 2023-10-05T21:05:25.792 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados-devel-2:17.2.6-1278.g487ffcb5.el8.x86_64 1/1 2023-10-05T21:05:26.698 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados-devel-2:17.2.6-1278.g487ffcb5.el8.x86_64 1/1 2023-10-05T21:05:26.698 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:05:26.698 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2023-10-05T21:05:26.698 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel-2:17.2.6-1278.g487ffcb5.el8.x86_64 2023-10-05T21:05:26.698 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:05:26.698 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-10-05T21:05:26.896 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install libcephfs2 2023-10-05T21:05:27.771 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:12:47 ago on Thu 05 Oct 2023 08:52:40 PM UTC. 2023-10-05T21:05:29.162 INFO:teuthology.orchestra.run.smithi167.stdout:Package libcephfs2-2:17.2.6-1278.g487ffcb5.el8.x86_64 is already installed. 2023-10-05T21:05:29.350 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-10-05T21:05:29.351 INFO:teuthology.orchestra.run.smithi167.stdout:Nothing to do. 2023-10-05T21:05:29.352 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-10-05T21:05:29.441 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install libcephfs-devel 2023-10-05T21:05:30.302 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:12:50 ago on Thu 05 Oct 2023 08:52:40 PM UTC. 2023-10-05T21:05:31.897 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-10-05T21:05:31.897 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:05:31.897 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repo Size 2023-10-05T21:05:31.897 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:05:31.898 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2023-10-05T21:05:31.898 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel x86_64 2:17.2.6-1278.g487ffcb5.el8 ceph 25 k 2023-10-05T21:05:31.898 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:05:31.898 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-10-05T21:05:31.898 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:05:31.898 INFO:teuthology.orchestra.run.smithi167.stdout:Install 1 Package 2023-10-05T21:05:31.898 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:05:31.899 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 25 k 2023-10-05T21:05:31.899 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 109 k 2023-10-05T21:05:31.899 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2023-10-05T21:05:32.122 INFO:teuthology.orchestra.run.smithi167.stdout:libcephfs-devel-17.2.6-1278.g487ffcb5.el8.x86_6 111 kB/s | 25 kB 00:00 2023-10-05T21:05:32.123 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2023-10-05T21:05:32.123 INFO:teuthology.orchestra.run.smithi167.stdout:Total 110 kB/s | 25 kB 00:00 2023-10-05T21:05:32.123 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2023-10-05T21:05:32.127 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2023-10-05T21:05:32.127 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2023-10-05T21:05:32.129 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2023-10-05T21:05:32.132 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2023-10-05T21:05:32.185 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2023-10-05T21:05:32.414 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs-devel-2:17.2.6-1278.g487ffcb5.el8.x86_64 1/1 2023-10-05T21:05:32.678 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephfs-devel-2:17.2.6-1278.g487ffcb5.el8.x86_64 1/1 2023-10-05T21:05:33.277 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs-devel-2:17.2.6-1278.g487ffcb5.el8.x86_64 1/1 2023-10-05T21:05:33.277 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:05:33.277 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2023-10-05T21:05:33.277 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel-2:17.2.6-1278.g487ffcb5.el8.x86_64 2023-10-05T21:05:33.277 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:05:33.277 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-10-05T21:05:33.386 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install librados2 2023-10-05T21:05:34.244 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:12:54 ago on Thu 05 Oct 2023 08:52:40 PM UTC. 2023-10-05T21:05:35.647 INFO:teuthology.orchestra.run.smithi167.stdout:Package librados2-2:17.2.6-1278.g487ffcb5.el8.x86_64 is already installed. 2023-10-05T21:05:35.841 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-10-05T21:05:35.842 INFO:teuthology.orchestra.run.smithi167.stdout:Nothing to do. 2023-10-05T21:05:35.842 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-10-05T21:05:35.944 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install librbd1 2023-10-05T21:05:36.808 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:12:56 ago on Thu 05 Oct 2023 08:52:40 PM UTC. 2023-10-05T21:05:38.206 INFO:teuthology.orchestra.run.smithi167.stdout:Package librbd1-2:17.2.6-1278.g487ffcb5.el8.x86_64 is already installed. 2023-10-05T21:05:38.396 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-10-05T21:05:38.397 INFO:teuthology.orchestra.run.smithi167.stdout:Nothing to do. 2023-10-05T21:05:38.397 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-10-05T21:05:38.495 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install python3-rados 2023-10-05T21:05:39.357 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:12:59 ago on Thu 05 Oct 2023 08:52:40 PM UTC. 2023-10-05T21:05:40.671 INFO:teuthology.orchestra.run.smithi167.stdout:Package python3-rados-2:17.2.6-1278.g487ffcb5.el8.x86_64 is already installed. 2023-10-05T21:05:40.861 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-10-05T21:05:40.862 INFO:teuthology.orchestra.run.smithi167.stdout:Nothing to do. 2023-10-05T21:05:40.862 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-10-05T21:05:40.966 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install python3-rgw 2023-10-05T21:05:41.834 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:13:01 ago on Thu 05 Oct 2023 08:52:40 PM UTC. 2023-10-05T21:05:43.238 INFO:teuthology.orchestra.run.smithi167.stdout:Package python3-rgw-2:17.2.6-1278.g487ffcb5.el8.x86_64 is already installed. 2023-10-05T21:05:43.427 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-10-05T21:05:43.428 INFO:teuthology.orchestra.run.smithi167.stdout:Nothing to do. 2023-10-05T21:05:43.428 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-10-05T21:05:43.537 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install python3-cephfs 2023-10-05T21:05:44.393 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:13:04 ago on Thu 05 Oct 2023 08:52:40 PM UTC. 2023-10-05T21:05:45.790 INFO:teuthology.orchestra.run.smithi167.stdout:Package python3-cephfs-2:17.2.6-1278.g487ffcb5.el8.x86_64 is already installed. 2023-10-05T21:05:45.977 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-10-05T21:05:45.979 INFO:teuthology.orchestra.run.smithi167.stdout:Nothing to do. 2023-10-05T21:05:45.979 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-10-05T21:05:46.077 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install python3-rbd 2023-10-05T21:05:46.950 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:13:06 ago on Thu 05 Oct 2023 08:52:40 PM UTC. 2023-10-05T21:05:48.348 INFO:teuthology.orchestra.run.smithi167.stdout:Package python3-rbd-2:17.2.6-1278.g487ffcb5.el8.x86_64 is already installed. 2023-10-05T21:05:48.539 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-10-05T21:05:48.540 INFO:teuthology.orchestra.run.smithi167.stdout:Nothing to do. 2023-10-05T21:05:48.540 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-10-05T21:05:48.635 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install rbd-fuse 2023-10-05T21:05:49.495 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:13:09 ago on Thu 05 Oct 2023 08:52:40 PM UTC. 2023-10-05T21:05:51.074 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-10-05T21:05:51.074 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:05:51.074 INFO:teuthology.orchestra.run.smithi167.stdout: Package Architecture Version Repository Size 2023-10-05T21:05:51.074 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:05:51.074 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2023-10-05T21:05:51.074 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse x86_64 2:17.2.6-1278.g487ffcb5.el8 ceph 90 k 2023-10-05T21:05:51.075 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:05:51.075 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-10-05T21:05:51.075 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:05:51.075 INFO:teuthology.orchestra.run.smithi167.stdout:Install 1 Package 2023-10-05T21:05:51.075 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:05:51.075 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 90 k 2023-10-05T21:05:51.075 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 264 k 2023-10-05T21:05:51.075 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2023-10-05T21:05:51.243 INFO:teuthology.orchestra.run.smithi167.stdout:rbd-fuse-17.2.6-1278.g487ffcb5.el8.x86_64.rpm 537 kB/s | 90 kB 00:00 2023-10-05T21:05:51.243 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2023-10-05T21:05:51.243 INFO:teuthology.orchestra.run.smithi167.stdout:Total 533 kB/s | 90 kB 00:00 2023-10-05T21:05:51.244 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2023-10-05T21:05:51.253 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2023-10-05T21:05:51.253 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2023-10-05T21:05:51.301 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2023-10-05T21:05:51.305 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2023-10-05T21:05:51.409 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2023-10-05T21:05:51.646 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-fuse-2:17.2.6-1278.g487ffcb5.el8.x86_64 1/1 2023-10-05T21:05:51.939 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: rbd-fuse-2:17.2.6-1278.g487ffcb5.el8.x86_64 1/1 2023-10-05T21:05:52.550 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-fuse-2:17.2.6-1278.g487ffcb5.el8.x86_64 1/1 2023-10-05T21:05:52.550 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:05:52.550 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2023-10-05T21:05:52.550 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse-2:17.2.6-1278.g487ffcb5.el8.x86_64 2023-10-05T21:05:52.550 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:05:52.550 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-10-05T21:05:52.741 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install rbd-mirror 2023-10-05T21:05:53.596 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:13:13 ago on Thu 05 Oct 2023 08:52:40 PM UTC. 2023-10-05T21:05:55.184 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-10-05T21:05:55.185 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:05:55.185 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repo Size 2023-10-05T21:05:55.185 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:05:55.185 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2023-10-05T21:05:55.185 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror x86_64 2:17.2.6-1278.g487ffcb5.el8 ceph 3.5 M 2023-10-05T21:05:55.185 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:05:55.185 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-10-05T21:05:55.185 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:05:55.185 INFO:teuthology.orchestra.run.smithi167.stdout:Install 1 Package 2023-10-05T21:05:55.185 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:05:55.186 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 3.5 M 2023-10-05T21:05:55.186 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 16 M 2023-10-05T21:05:55.186 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2023-10-05T21:05:55.571 INFO:teuthology.orchestra.run.smithi167.stdout:rbd-mirror-17.2.6-1278.g487ffcb5.el8.x86_64.rpm 9.0 MB/s | 3.5 MB 00:00 2023-10-05T21:05:55.572 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2023-10-05T21:05:55.572 INFO:teuthology.orchestra.run.smithi167.stdout:Total 8.9 MB/s | 3.5 MB 00:00 2023-10-05T21:05:55.572 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2023-10-05T21:05:55.583 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2023-10-05T21:05:55.584 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2023-10-05T21:05:55.646 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2023-10-05T21:05:55.649 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2023-10-05T21:05:56.084 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2023-10-05T21:05:56.255 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-mirror-2:17.2.6-1278.g487ffcb5.el8.x86_64 1/1 2023-10-05T21:05:56.669 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1278.g487ffcb5.el8.x86_64 1/1 2023-10-05T21:05:57.286 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-mirror-2:17.2.6-1278.g487ffcb5.el8.x86_64 1/1 2023-10-05T21:05:57.286 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:05:57.286 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2023-10-05T21:05:57.286 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror-2:17.2.6-1278.g487ffcb5.el8.x86_64 2023-10-05T21:05:57.286 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:05:57.287 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-10-05T21:05:57.473 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install rbd-nbd 2023-10-05T21:05:58.343 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:13:18 ago on Thu 05 Oct 2023 08:52:40 PM UTC. 2023-10-05T21:05:59.943 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2023-10-05T21:05:59.944 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:05:59.944 INFO:teuthology.orchestra.run.smithi167.stdout: Package Architecture Version Repository Size 2023-10-05T21:05:59.944 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:05:59.944 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2023-10-05T21:05:59.944 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd x86_64 2:17.2.6-1278.g487ffcb5.el8 ceph 173 k 2023-10-05T21:05:59.944 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:05:59.944 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2023-10-05T21:05:59.945 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2023-10-05T21:05:59.945 INFO:teuthology.orchestra.run.smithi167.stdout:Install 1 Package 2023-10-05T21:05:59.945 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:05:59.945 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 173 k 2023-10-05T21:05:59.945 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 552 k 2023-10-05T21:05:59.945 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2023-10-05T21:06:00.133 INFO:teuthology.orchestra.run.smithi167.stdout:rbd-nbd-17.2.6-1278.g487ffcb5.el8.x86_64.rpm 918 kB/s | 173 kB 00:00 2023-10-05T21:06:00.134 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2023-10-05T21:06:00.134 INFO:teuthology.orchestra.run.smithi167.stdout:Total 912 kB/s | 173 kB 00:00 2023-10-05T21:06:00.134 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2023-10-05T21:06:00.145 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2023-10-05T21:06:00.145 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2023-10-05T21:06:00.194 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2023-10-05T21:06:00.197 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2023-10-05T21:06:00.310 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2023-10-05T21:06:00.511 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-nbd-2:17.2.6-1278.g487ffcb5.el8.x86_64 1/1 2023-10-05T21:06:00.817 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: rbd-nbd-2:17.2.6-1278.g487ffcb5.el8.x86_64 1/1 2023-10-05T21:06:01.434 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-nbd-2:17.2.6-1278.g487ffcb5.el8.x86_64 1/1 2023-10-05T21:06:01.434 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:06:01.434 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2023-10-05T21:06:01.434 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd-2:17.2.6-1278.g487ffcb5.el8.x86_64 2023-10-05T21:06:01.434 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:06:01.434 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2023-10-05T21:06:01.626 DEBUG:teuthology.parallel:result is None 2023-10-05T21:06:01.627 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=487ffcb5ddb60d29c5688d1a457e812b1b697b90 2023-10-05T21:06:01.778 DEBUG:teuthology.orchestra.run.smithi167:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-05T21:06:01.801 INFO:teuthology.orchestra.run.smithi167.stdout:17.2.6-1278.g487ffcb5.el8 2023-10-05T21:06:01.801 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1278.g487ffcb5.el8 2023-10-05T21:06:01.801 INFO:teuthology.task.install:The correct ceph version 17.2.6-1278.g487ffcb5 is installed. 2023-10-05T21:06:01.802 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-05T21:06:01.802 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-05T21:06:01.802 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-05T21:06:01.869 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-05T21:06:01.869 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-05T21:06:01.869 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/daemon-helper 2023-10-05T21:06:01.936 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-05T21:06:02.002 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-05T21:06:02.003 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-05T21:06:02.003 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-05T21:06:02.070 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-05T21:06:02.137 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-05T21:06:02.137 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-05T21:06:02.137 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/stdin-killer 2023-10-05T21:06:02.204 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-05T21:06:02.271 INFO:teuthology.run_tasks:Running task ceph... 2023-10-05T21:06:02.329 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-05T21:06:02.330 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 777 /var/log/ceph 2023-10-05T21:06:02.356 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-05T21:06:02.356 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-05T21:06:02.424 INFO:tasks.ceph:Creating extra log directories... 2023-10-05T21:06:02.424 DEBUG:teuthology.orchestra.run.smithi167:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-05T21:06:02.499 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-05T21:06:02.499 INFO:tasks.ceph:config {'conf': {'client': {'rbd default format': 1}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(CACHE_POOL_NO_HIT_SET\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-05T21:06:02.499 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-10-05_14:31:59-rbd-wip-yuri2-testing-2023-10-04-1333-quincy-distro-default-smithi/7411791', 'branch': 'wip-yuri2-testing-2023-10-04-1333-quincy', 'description': 'rbd/cli_v1/{base/install clusters/{fixed-1 openstack} conf/{disable-pool-app} features/format-1 msgr-failures/few objectstore/bluestore-comp-zlib pool/small-cache-pool supported-random-distro$/{centos_8} workloads/rbd_cli_generic}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7411791', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'yuriw-2023-10-05_14:31:59-rbd-wip-yuri2-testing-2023-10-04-1333-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 3, 'size': 30}}], 'os_type': 'centos', 'os_version': '8.stream', 'overrides': {'admin_socket': {'branch': 'wip-yuri2-testing-2023-10-04-1333-quincy'}, 'ceph': {'conf': {'client': {'rbd default format': 1}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(CACHE_POOL_NO_HIT_SET\\)'], 'sha1': '487ffcb5ddb60d29c5688d1a457e812b1b697b90'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'global': {'osd crush chooseleaf type': 0, 'osd pool default pg num': 128, 'osd pool default pgp num': 128, 'osd pool default size': 2}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '487ffcb5ddb60d29c5688d1a457e812b1b697b90'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri2-testing-2023-10-04-1333-quincy', 'sha1': '487ffcb5ddb60d29c5688d1a457e812b1b697b90'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0']], 'seed': 652, 'sha1': '487ffcb5ddb60d29c5688d1a457e812b1b697b90', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'rbd', 'suite_branch': 'wip-yuri2-testing-2023-10-04-1333-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_487ffcb5ddb60d29c5688d1a457e812b1b697b90/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '487ffcb5ddb60d29c5688d1a457e812b1b697b90', 'targets': {'smithi167.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBIRuzAhk+yMzyhTJFKgQFK82kkX2FDeRQJVhUXDDAz8byAnwnstl+xvDmRzlH0emr8zzGNpH4zXzX5NUQczVBk='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'exec': {'client.0': ['sudo ceph osd pool create cache 4', 'sudo ceph osd tier add rbd cache', 'sudo ceph osd tier cache-mode cache writeback', 'sudo ceph osd tier set-overlay rbd cache', 'sudo ceph osd pool set cache hit_set_type bloom', 'sudo ceph osd pool set cache hit_set_count 8', 'sudo ceph osd pool set cache hit_set_period 60', 'sudo ceph osd pool set cache target_max_objects 250']}}, {'workunit': {'clients': {'client.0': ['rbd/cli_generic.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-10-05_14:31:59', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-05T21:06:02.499 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-05T21:06:02.548 DEBUG:teuthology.orchestra.run.smithi167:> sudo install -d -m0777 -- /var/run/ceph 2023-10-05T21:06:02.615 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-05T21:06:02.616 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/scratch_devs of=/dev/stdout 2023-10-05T21:06:02.674 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-05T21:06:02.674 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_1 2023-10-05T21:06:02.731 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-05T21:06:02.732 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-05T21:06:02.732 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 424056 Links: 1 2023-10-05T21:06:02.732 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-05T21:06:02.732 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2023-10-05T21:06:02.732 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2023-10-05 21:05:56.417065043 +0000 2023-10-05T21:06:02.732 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2023-10-05 20:32:43.823988737 +0000 2023-10-05T21:06:02.732 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2023-10-05 20:32:43.823988737 +0000 2023-10-05T21:06:02.732 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2023-10-05T21:06:02.733 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-05T21:06:02.797 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2023-10-05T21:06:02.798 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2023-10-05T21:06:02.798 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000205089 s, 2.5 MB/s 2023-10-05T21:06:02.799 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-05T21:06:02.858 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_2 2023-10-05T21:06:02.916 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-05T21:06:02.916 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-05T21:06:02.916 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 423730 Links: 1 2023-10-05T21:06:02.916 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-05T21:06:02.916 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2023-10-05T21:06:02.916 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2023-10-05 21:05:56.417065043 +0000 2023-10-05T21:06:02.916 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2023-10-05 20:32:44.337979327 +0000 2023-10-05T21:06:02.916 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2023-10-05 20:32:44.337979327 +0000 2023-10-05T21:06:02.917 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2023-10-05T21:06:02.917 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-05T21:06:02.981 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2023-10-05T21:06:02.981 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2023-10-05T21:06:02.981 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000219427 s, 2.3 MB/s 2023-10-05T21:06:02.982 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-05T21:06:03.041 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_3 2023-10-05T21:06:03.099 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-05T21:06:03.100 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-05T21:06:03.100 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 427247 Links: 1 2023-10-05T21:06:03.100 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-05T21:06:03.100 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2023-10-05T21:06:03.100 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2023-10-05 21:05:56.417065043 +0000 2023-10-05T21:06:03.100 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2023-10-05 20:32:44.858969788 +0000 2023-10-05T21:06:03.100 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2023-10-05 20:32:44.858969788 +0000 2023-10-05T21:06:03.100 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2023-10-05T21:06:03.101 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-05T21:06:03.165 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2023-10-05T21:06:03.165 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2023-10-05T21:06:03.165 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000233988 s, 2.2 MB/s 2023-10-05T21:06:03.166 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-05T21:06:03.225 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_4 2023-10-05T21:06:03.284 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-05T21:06:03.284 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-05T21:06:03.284 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 424714 Links: 1 2023-10-05T21:06:03.284 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-05T21:06:03.284 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2023-10-05T21:06:03.284 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2023-10-05 21:05:56.417065043 +0000 2023-10-05T21:06:03.284 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2023-10-05 20:32:45.352960743 +0000 2023-10-05T21:06:03.285 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2023-10-05 20:32:45.352960743 +0000 2023-10-05T21:06:03.285 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2023-10-05T21:06:03.285 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-05T21:06:03.349 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2023-10-05T21:06:03.349 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2023-10-05T21:06:03.349 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000233313 s, 2.2 MB/s 2023-10-05T21:06:03.350 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-05T21:06:03.409 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2023-10-05T21:06:03.410 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi167.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2023-10-05T21:06:03.410 INFO:tasks.ceph:Generating config... 2023-10-05T21:06:03.411 INFO:tasks.ceph:[client] rbd default format = 1 2023-10-05T21:06:03.411 INFO:tasks.ceph:[global] mon client directed command retry = 5 2023-10-05T21:06:03.411 INFO:tasks.ceph:[global] mon warn on pool no app = False 2023-10-05T21:06:03.412 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2023-10-05T21:06:03.412 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-05T21:06:03.412 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-05T21:06:03.412 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-05T21:06:03.412 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-05T21:06:03.412 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-05T21:06:03.412 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-05T21:06:03.412 INFO:tasks.ceph:[osd] bluestore compression algorithm = zlib 2023-10-05T21:06:03.413 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2023-10-05T21:06:03.413 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-05T21:06:03.413 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-10-05T21:06:03.413 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-10-05T21:06:03.413 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-05T21:06:03.413 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-05T21:06:03.413 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-10-05T21:06:03.414 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-05T21:06:03.414 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-05T21:06:03.414 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-05T21:06:03.414 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-05T21:06:03.414 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-05T21:06:03.414 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2023-10-05T21:06:03.414 INFO:tasks.ceph:Setting up mon.a... 2023-10-05T21:06:03.415 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-05T21:06:03.506 INFO:teuthology.orchestra.run.smithi167.stdout:creating /etc/ceph/ceph.keyring 2023-10-05T21:06:03.508 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-05T21:06:03.550 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-05T21:06:03.616 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.167')] 2023-10-05T21:06:03.617 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.167', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd default format': 1}, 'mon.a': {}} 2023-10-05T21:06:03.618 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-05T21:06:03.618 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-05T21:06:03.675 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.167 --print /home/ubuntu/cephtest/ceph.monmap 2023-10-05T21:06:03.754 INFO:teuthology.orchestra.run.smithi167.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-05T21:06:03.754 INFO:teuthology.orchestra.run.smithi167.stdout:monmaptool: generated fsid 18db5e25-e3f3-4284-9e58-5e5554ae5140 2023-10-05T21:06:03.754 INFO:teuthology.orchestra.run.smithi167.stdout:setting min_mon_release = octopus 2023-10-05T21:06:03.755 INFO:teuthology.orchestra.run.smithi167.stdout:epoch 0 2023-10-05T21:06:03.755 INFO:teuthology.orchestra.run.smithi167.stdout:fsid 18db5e25-e3f3-4284-9e58-5e5554ae5140 2023-10-05T21:06:03.755 INFO:teuthology.orchestra.run.smithi167.stdout:last_changed 2023-10-05T21:06:03.753603+0000 2023-10-05T21:06:03.755 INFO:teuthology.orchestra.run.smithi167.stdout:created 2023-10-05T21:06:03.753603+0000 2023-10-05T21:06:03.755 INFO:teuthology.orchestra.run.smithi167.stdout:min_mon_release 15 (octopus) 2023-10-05T21:06:03.755 INFO:teuthology.orchestra.run.smithi167.stdout:election_strategy: 1 2023-10-05T21:06:03.755 INFO:teuthology.orchestra.run.smithi167.stdout:0: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.a 2023-10-05T21:06:03.755 INFO:teuthology.orchestra.run.smithi167.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2023-10-05T21:06:03.757 DEBUG:teuthology.orchestra.run.smithi167:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-05T21:06:03.814 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 18db5e25-e3f3-4284-9e58-5e5554ae5140... 2023-10-05T21:06:03.815 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-05T21:06:03.900 INFO:teuthology.orchestra.run.smithi167.stdout:[global] 2023-10-05T21:06:03.900 INFO:teuthology.orchestra.run.smithi167.stdout: chdir = "" 2023-10-05T21:06:03.900 INFO:teuthology.orchestra.run.smithi167.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-05T21:06:03.900 INFO:teuthology.orchestra.run.smithi167.stdout: auth supported = cephx 2023-10-05T21:06:03.901 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:06:03.901 INFO:teuthology.orchestra.run.smithi167.stdout: filestore xattr use omap = true 2023-10-05T21:06:03.901 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:06:03.901 INFO:teuthology.orchestra.run.smithi167.stdout: mon clock drift allowed = 1.000 2023-10-05T21:06:03.901 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:06:03.901 INFO:teuthology.orchestra.run.smithi167.stdout: osd crush chooseleaf type = 0 2023-10-05T21:06:03.901 INFO:teuthology.orchestra.run.smithi167.stdout: auth debug = true 2023-10-05T21:06:03.901 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:06:03.901 INFO:teuthology.orchestra.run.smithi167.stdout: ms die on old message = true 2023-10-05T21:06:03.901 INFO:teuthology.orchestra.run.smithi167.stdout: ms die on bug = true 2023-10-05T21:06:03.901 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:06:03.902 INFO:teuthology.orchestra.run.smithi167.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-05T21:06:03.902 INFO:teuthology.orchestra.run.smithi167.stdout: mon pg warn max object skew = 0 2023-10-05T21:06:03.902 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:06:03.902 INFO:teuthology.orchestra.run.smithi167.stdout: # disable pg_autoscaler by default for new pools 2023-10-05T21:06:03.902 INFO:teuthology.orchestra.run.smithi167.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-05T21:06:03.902 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:06:03.902 INFO:teuthology.orchestra.run.smithi167.stdout: osd pool default size = 2 2023-10-05T21:06:03.902 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:06:03.902 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd allow primary affinity = true 2023-10-05T21:06:03.902 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd allow pg remap = true 2023-10-05T21:06:03.902 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on legacy crush tunables = false 2023-10-05T21:06:03.903 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on crush straw calc version zero = false 2023-10-05T21:06:03.903 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on no sortbitwise = false 2023-10-05T21:06:03.903 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on osd down out interval zero = false 2023-10-05T21:06:03.903 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on too few osds = false 2023-10-05T21:06:03.903 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-05T21:06:03.903 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-05T21:06:03.903 INFO:teuthology.orchestra.run.smithi167.stdout: mon_allow_pool_size_one = true 2023-10-05T21:06:03.903 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:06:03.903 INFO:teuthology.orchestra.run.smithi167.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-05T21:06:03.903 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:06:03.903 INFO:teuthology.orchestra.run.smithi167.stdout: osd default data pool replay window = 5 2023-10-05T21:06:03.903 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:06:03.904 INFO:teuthology.orchestra.run.smithi167.stdout: mon allow pool delete = true 2023-10-05T21:06:03.904 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:06:03.904 INFO:teuthology.orchestra.run.smithi167.stdout: mon cluster log file level = debug 2023-10-05T21:06:03.904 INFO:teuthology.orchestra.run.smithi167.stdout: debug asserts on shutdown = true 2023-10-05T21:06:03.904 INFO:teuthology.orchestra.run.smithi167.stdout: mon health detail to clog = false 2023-10-05T21:06:03.904 INFO:teuthology.orchestra.run.smithi167.stdout: mon host = 172.21.15.167 2023-10-05T21:06:03.904 INFO:teuthology.orchestra.run.smithi167.stdout: mon client directed command retry = 5 2023-10-05T21:06:03.904 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on pool no app = False 2023-10-05T21:06:03.904 INFO:teuthology.orchestra.run.smithi167.stdout: ms inject socket failures = 5000 2023-10-05T21:06:03.904 INFO:teuthology.orchestra.run.smithi167.stdout: fsid = 18db5e25-e3f3-4284-9e58-5e5554ae5140 2023-10-05T21:06:03.904 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:06:03.904 INFO:teuthology.orchestra.run.smithi167.stdout:[osd] 2023-10-05T21:06:03.905 INFO:teuthology.orchestra.run.smithi167.stdout: osd journal size = 100 2023-10-05T21:06:03.905 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:06:03.905 INFO:teuthology.orchestra.run.smithi167.stdout: osd scrub load threshold = 5.0 2023-10-05T21:06:03.905 INFO:teuthology.orchestra.run.smithi167.stdout: osd scrub max interval = 600 2023-10-05T21:06:03.905 INFO:teuthology.orchestra.run.smithi167.stdout: osd mclock profile = high_recovery_ops 2023-10-05T21:06:03.905 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:06:03.905 INFO:teuthology.orchestra.run.smithi167.stdout: osd recover clone overlap = true 2023-10-05T21:06:03.905 INFO:teuthology.orchestra.run.smithi167.stdout: osd recovery max chunk = 1048576 2023-10-05T21:06:03.905 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:06:03.905 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug shutdown = true 2023-10-05T21:06:03.905 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug op order = true 2023-10-05T21:06:03.905 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug verify stray on activate = true 2023-10-05T21:06:03.906 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:06:03.906 INFO:teuthology.orchestra.run.smithi167.stdout: osd open classes on start = true 2023-10-05T21:06:03.906 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug pg log writeout = true 2023-10-05T21:06:03.906 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:06:03.906 INFO:teuthology.orchestra.run.smithi167.stdout: osd deep scrub update digest min age = 30 2023-10-05T21:06:03.906 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:06:03.906 INFO:teuthology.orchestra.run.smithi167.stdout: osd map max advance = 10 2023-10-05T21:06:03.906 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:06:03.906 INFO:teuthology.orchestra.run.smithi167.stdout: journal zero on create = true 2023-10-05T21:06:03.906 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:06:03.906 INFO:teuthology.orchestra.run.smithi167.stdout: filestore ondisk finisher threads = 3 2023-10-05T21:06:03.906 INFO:teuthology.orchestra.run.smithi167.stdout: filestore apply finisher threads = 3 2023-10-05T21:06:03.907 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:06:03.907 INFO:teuthology.orchestra.run.smithi167.stdout: bdev debug aio = true 2023-10-05T21:06:03.907 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug misdirected ops = true 2023-10-05T21:06:03.907 INFO:teuthology.orchestra.run.smithi167.stdout: bluestore block size = 96636764160 2023-10-05T21:06:03.907 INFO:teuthology.orchestra.run.smithi167.stdout: bluestore compression algorithm = zlib 2023-10-05T21:06:03.907 INFO:teuthology.orchestra.run.smithi167.stdout: bluestore compression mode = aggressive 2023-10-05T21:06:03.907 INFO:teuthology.orchestra.run.smithi167.stdout: bluestore fsck on mount = True 2023-10-05T21:06:03.907 INFO:teuthology.orchestra.run.smithi167.stdout: debug bluefs = 1/20 2023-10-05T21:06:03.907 INFO:teuthology.orchestra.run.smithi167.stdout: debug bluestore = 1/20 2023-10-05T21:06:03.907 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2023-10-05T21:06:03.907 INFO:teuthology.orchestra.run.smithi167.stdout: debug osd = 20 2023-10-05T21:06:03.907 INFO:teuthology.orchestra.run.smithi167.stdout: debug rocksdb = 4/10 2023-10-05T21:06:03.907 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd backfillfull_ratio = 0.85 2023-10-05T21:06:03.908 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd full ratio = 0.9 2023-10-05T21:06:03.908 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd nearfull ratio = 0.8 2023-10-05T21:06:03.908 INFO:teuthology.orchestra.run.smithi167.stdout: osd failsafe full ratio = 0.95 2023-10-05T21:06:03.908 INFO:teuthology.orchestra.run.smithi167.stdout: osd objectstore = bluestore 2023-10-05T21:06:03.908 INFO:teuthology.orchestra.run.smithi167.stdout: osd shutdown pgref assert = True 2023-10-05T21:06:03.908 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:06:03.908 INFO:teuthology.orchestra.run.smithi167.stdout:[mgr] 2023-10-05T21:06:03.908 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2023-10-05T21:06:03.908 INFO:teuthology.orchestra.run.smithi167.stdout: debug mgr = 20 2023-10-05T21:06:03.908 INFO:teuthology.orchestra.run.smithi167.stdout: debug mon = 20 2023-10-05T21:06:03.908 INFO:teuthology.orchestra.run.smithi167.stdout: debug auth = 20 2023-10-05T21:06:03.908 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min pgs per osd = 4 2023-10-05T21:06:03.909 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min bytes per osd = 10 2023-10-05T21:06:03.909 INFO:teuthology.orchestra.run.smithi167.stdout: mgr/telemetry/nag = false 2023-10-05T21:06:03.909 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:06:03.909 INFO:teuthology.orchestra.run.smithi167.stdout:[mon] 2023-10-05T21:06:03.909 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2023-10-05T21:06:03.909 INFO:teuthology.orchestra.run.smithi167.stdout: debug mon = 20 2023-10-05T21:06:03.909 INFO:teuthology.orchestra.run.smithi167.stdout: debug paxos = 20 2023-10-05T21:06:03.909 INFO:teuthology.orchestra.run.smithi167.stdout: debug auth = 20 2023-10-05T21:06:03.909 INFO:teuthology.orchestra.run.smithi167.stdout: mon data avail warn = 5 2023-10-05T21:06:03.909 INFO:teuthology.orchestra.run.smithi167.stdout: mon mgr mkfs grace = 240 2023-10-05T21:06:03.910 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min pgs per osd = 4 2023-10-05T21:06:03.910 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd reporter subtree level = osd 2023-10-05T21:06:03.910 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd prime pg temp = true 2023-10-05T21:06:03.910 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min bytes per osd = 10 2023-10-05T21:06:03.910 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:06:03.910 INFO:teuthology.orchestra.run.smithi167.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-05T21:06:03.910 INFO:teuthology.orchestra.run.smithi167.stdout: auth mon ticket ttl = 660 # 11m 2023-10-05T21:06:03.910 INFO:teuthology.orchestra.run.smithi167.stdout: auth service ticket ttl = 240 # 4m 2023-10-05T21:06:03.911 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:06:03.911 INFO:teuthology.orchestra.run.smithi167.stdout: # don't complain about insecure global_id in the test suite 2023-10-05T21:06:03.911 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-05T21:06:03.911 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-05T21:06:03.911 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:06:03.911 INFO:teuthology.orchestra.run.smithi167.stdout: # 1m isn't quite enough 2023-10-05T21:06:03.911 INFO:teuthology.orchestra.run.smithi167.stdout: mon_down_mkfs_grace = 2m 2023-10-05T21:06:03.911 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:06:03.911 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_filestore_osds = false 2023-10-05T21:06:03.911 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:06:03.912 INFO:teuthology.orchestra.run.smithi167.stdout:[client] 2023-10-05T21:06:03.912 INFO:teuthology.orchestra.run.smithi167.stdout: rgw cache enabled = true 2023-10-05T21:06:03.912 INFO:teuthology.orchestra.run.smithi167.stdout: rgw enable ops log = true 2023-10-05T21:06:03.912 INFO:teuthology.orchestra.run.smithi167.stdout: rgw enable usage log = true 2023-10-05T21:06:03.912 INFO:teuthology.orchestra.run.smithi167.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-05T21:06:03.912 INFO:teuthology.orchestra.run.smithi167.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-05T21:06:03.912 INFO:teuthology.orchestra.run.smithi167.stdout: rbd default format = 1 2023-10-05T21:06:03.912 INFO:teuthology.orchestra.run.smithi167.stdout:[mon.a] 2023-10-05T21:06:03.913 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-05T21:06:03.913 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-05T21:06:04.007 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-05T21:06:04.007 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-05T21:06:04.007 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-05T21:06:04.026 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-05T21:06:04.026 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-05T21:06:04.083 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi167.front.sepia.ceph.com 2023-10-05T21:06:04.083 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-05T21:06:04.084 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-05T21:06:04.084 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-05T21:06:04.160 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-05T21:06:04.160 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-05T21:06:04.218 INFO:tasks.ceph:Setting up mon nodes... 2023-10-05T21:06:04.219 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-05T21:06:04.219 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-05T21:06:04.309 INFO:teuthology.orchestra.run.smithi167.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-05T21:06:04.311 INFO:tasks.ceph:Setting up mds nodes... 2023-10-05T21:06:04.311 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-05T21:06:04.312 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-05T21:06:04.352 INFO:teuthology.orchestra.run.smithi167.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-05T21:06:04.363 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-05T21:06:04.364 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi167.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2023-10-05T21:06:04.364 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-05T21:06:04.432 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2023-10-05T21:06:04.432 INFO:tasks.ceph:role: osd.0 2023-10-05T21:06:04.432 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi167.front.sepia.ceph.com 2023-10-05T21:06:04.432 DEBUG:teuthology.orchestra.run.smithi167:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-05T21:06:04.506 INFO:teuthology.orchestra.run.smithi167.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-05T21:06:04.506 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-05T21:06:04.507 INFO:teuthology.orchestra.run.smithi167.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-05T21:06:04.507 INFO:teuthology.orchestra.run.smithi167.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-05T21:06:04.507 INFO:teuthology.orchestra.run.smithi167.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-05T21:06:04.507 INFO:teuthology.orchestra.run.smithi167.stdout: = sunit=0 swidth=0 blks 2023-10-05T21:06:04.507 INFO:teuthology.orchestra.run.smithi167.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-05T21:06:04.507 INFO:teuthology.orchestra.run.smithi167.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-05T21:06:04.507 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-05T21:06:04.507 INFO:teuthology.orchestra.run.smithi167.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-05T21:06:04.928 INFO:teuthology.orchestra.run.smithi167.stdout:Discarding blocks...Done. 2023-10-05T21:06:04.930 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi167.front.sepia.ceph.com -o noatime 2023-10-05T21:06:04.930 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-05T21:06:05.090 DEBUG:teuthology.orchestra.run.smithi167:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-05T21:06:05.125 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-05T21:06:05.194 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2023-10-05T21:06:05.194 INFO:tasks.ceph:role: osd.1 2023-10-05T21:06:05.194 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi167.front.sepia.ceph.com 2023-10-05T21:06:05.194 DEBUG:teuthology.orchestra.run.smithi167:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-05T21:06:05.268 INFO:teuthology.orchestra.run.smithi167.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-05T21:06:05.269 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-05T21:06:05.269 INFO:teuthology.orchestra.run.smithi167.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-05T21:06:05.269 INFO:teuthology.orchestra.run.smithi167.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-05T21:06:05.269 INFO:teuthology.orchestra.run.smithi167.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-05T21:06:05.269 INFO:teuthology.orchestra.run.smithi167.stdout: = sunit=0 swidth=0 blks 2023-10-05T21:06:05.270 INFO:teuthology.orchestra.run.smithi167.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-05T21:06:05.270 INFO:teuthology.orchestra.run.smithi167.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-05T21:06:05.270 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-05T21:06:05.270 INFO:teuthology.orchestra.run.smithi167.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-05T21:06:05.714 INFO:teuthology.orchestra.run.smithi167.stdout:Discarding blocks...Done. 2023-10-05T21:06:05.715 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi167.front.sepia.ceph.com -o noatime 2023-10-05T21:06:05.716 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-05T21:06:05.832 DEBUG:teuthology.orchestra.run.smithi167:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-05T21:06:05.864 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-05T21:06:05.931 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2023-10-05T21:06:05.931 INFO:tasks.ceph:role: osd.2 2023-10-05T21:06:05.931 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi167.front.sepia.ceph.com 2023-10-05T21:06:05.931 DEBUG:teuthology.orchestra.run.smithi167:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-05T21:06:06.005 INFO:teuthology.orchestra.run.smithi167.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-05T21:06:06.005 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-05T21:06:06.005 INFO:teuthology.orchestra.run.smithi167.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-05T21:06:06.005 INFO:teuthology.orchestra.run.smithi167.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-05T21:06:06.005 INFO:teuthology.orchestra.run.smithi167.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-05T21:06:06.005 INFO:teuthology.orchestra.run.smithi167.stdout: = sunit=0 swidth=0 blks 2023-10-05T21:06:06.006 INFO:teuthology.orchestra.run.smithi167.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-05T21:06:06.006 INFO:teuthology.orchestra.run.smithi167.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-05T21:06:06.006 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-05T21:06:06.006 INFO:teuthology.orchestra.run.smithi167.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-05T21:06:06.447 INFO:teuthology.orchestra.run.smithi167.stdout:Discarding blocks...Done. 2023-10-05T21:06:06.448 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi167.front.sepia.ceph.com -o noatime 2023-10-05T21:06:06.448 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-05T21:06:06.607 DEBUG:teuthology.orchestra.run.smithi167:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-05T21:06:06.639 DEBUG:teuthology.orchestra.run.smithi167:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-05T21:06:06.736 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-05T21:06:06.734+0000 7faabe669540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-05T21:06:06.739 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-05T21:06:06.737+0000 7faabe669540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-05T21:06:06.739 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-05T21:06:06.737+0000 7faabe669540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-05T21:06:06.739 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-05T21:06:06.737+0000 7faabe669540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-05T21:06:06.739 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-05T21:06:06.737+0000 7faabe669540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-05T21:06:06.742 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-05T21:06:06.741+0000 7faabe669540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-05T21:06:07.268 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-05T21:06:07.266+0000 7faabe669540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-05T21:06:08.032 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-05T21:06:08.031+0000 7faabe669540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-05T21:06:08.795 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-05T21:06:08.793+0000 7faabe669540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-05T21:06:09.590 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-05T21:06:09.622 DEBUG:teuthology.orchestra.run.smithi167:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-05T21:06:09.709 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-05T21:06:09.707+0000 7fb9d5498540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-05T21:06:09.709 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-05T21:06:09.707+0000 7fb9d5498540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-05T21:06:09.709 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-05T21:06:09.707+0000 7fb9d5498540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-05T21:06:09.709 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-05T21:06:09.707+0000 7fb9d5498540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-05T21:06:09.709 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-05T21:06:09.707+0000 7fb9d5498540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-05T21:06:09.710 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-05T21:06:09.708+0000 7fb9d5498540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-05T21:06:10.233 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-05T21:06:10.231+0000 7fb9d5498540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-05T21:06:10.996 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-05T21:06:10.994+0000 7fb9d5498540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-05T21:06:11.762 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-05T21:06:11.760+0000 7fb9d5498540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-05T21:06:12.554 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-05T21:06:12.585 DEBUG:teuthology.orchestra.run.smithi167:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-05T21:06:12.672 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-05T21:06:12.670+0000 7f1aa2b44540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-05T21:06:12.672 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-05T21:06:12.670+0000 7f1aa2b44540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-05T21:06:12.672 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-05T21:06:12.670+0000 7f1aa2b44540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-05T21:06:12.672 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-05T21:06:12.670+0000 7f1aa2b44540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-05T21:06:12.672 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-05T21:06:12.670+0000 7f1aa2b44540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-05T21:06:12.673 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-05T21:06:12.671+0000 7f1aa2b44540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-05T21:06:13.196 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-05T21:06:13.194+0000 7f1aa2b44540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-05T21:06:13.957 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-05T21:06:13.955+0000 7f1aa2b44540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-05T21:06:14.720 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-05T21:06:14.718+0000 7f1aa2b44540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-05T21:06:15.509 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-05T21:06:15.539 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-05T21:06:15.539 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-05T21:06:15.539 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-05T21:06:15.608 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-05T21:06:15.608 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-05T21:06:15.677 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-05T21:06:15.677 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-05T21:06:15.744 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-05T21:06:15.744 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-05T21:06:15.812 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-05T21:06:15.812 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-05T21:06:15.869 INFO:tasks.ceph:Adding keys to all mons... 2023-10-05T21:06:15.870 DEBUG:teuthology.orchestra.run.smithi167:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-05T21:06:15.935 INFO:teuthology.orchestra.run.smithi167.stdout:[mgr.x] 2023-10-05T21:06:15.936 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQA8JR9lemRtEhAA304KNPRaqmRhJMJvCnoHKQ== 2023-10-05T21:06:15.936 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.0] 2023-10-05T21:06:15.936 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQA+JR9lQdjfKxAAm5svEsCpa/qLjuZPrLY0Pg== 2023-10-05T21:06:15.936 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.1] 2023-10-05T21:06:15.936 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQBBJR9lp9w7KhAAtNuXvMo7sL6ra5pkRwMuQA== 2023-10-05T21:06:15.936 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.2] 2023-10-05T21:06:15.936 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQBEJR9lQZIHKBAAuBy/g/PXmixrlGtota4BEQ== 2023-10-05T21:06:15.937 INFO:teuthology.orchestra.run.smithi167.stdout:[client.0] 2023-10-05T21:06:15.937 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQA8JR9log73FBAAbhyi6ljL8IyvyOwoyI5CHg== 2023-10-05T21:06:15.938 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-05T21:06:16.028 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-05T21:06:16.079 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-05T21:06:16.129 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-05T21:06:16.180 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-05T21:06:16.234 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-05T21:06:16.234 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-05T21:06:16.262 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-05T21:06:16.359 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-05T21:06:16.385 DEBUG:teuthology.orchestra.run.smithi167:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-05T21:06:16.443 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-05T21:06:16.444 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-05T21:06:16.444 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-05T21:06:16.486 INFO:tasks.ceph.mon.a:Started 2023-10-05T21:06:16.486 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-05T21:06:16.486 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-05T21:06:16.486 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-05T21:06:16.488 INFO:tasks.ceph.mgr.x:Started 2023-10-05T21:06:16.488 INFO:tasks.ceph:Setting crush tunables to default 2023-10-05T21:06:16.489 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd crush tunables default 2023-10-05T21:06:16.690 INFO:teuthology.orchestra.run.smithi167.stderr:adjusted tunables profile to default 2023-10-05T21:06:16.701 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-05T21:06:16.701 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-05T21:06:16.701 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-05T21:06:16.728 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-05T21:06:16.728 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-05T21:06:16.795 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-05T21:06:16.795 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-05T21:06:16.863 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new ab74b673-a624-4ec2-ad02-409b63519337 0 2023-10-05T21:06:16.882 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-05T21:06:16.881+0000 7efdae9541c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-05T21:06:17.058 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-05T21:06:17.056+0000 7efdae9541c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-05T21:06:17.138 INFO:teuthology.orchestra.run.smithi167.stdout:0 2023-10-05T21:06:17.151 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new 77f471b1-fe86-4b52-8091-01e503bab3c0 1 2023-10-05T21:06:17.378 INFO:teuthology.orchestra.run.smithi167.stdout:1 2023-10-05T21:06:17.391 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new 2e146899-f47c-4e3e-aeb0-737ab3b287d8 2 2023-10-05T21:06:17.423 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-05T21:06:17.421+0000 7efdae9541c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-05T21:06:17.631 INFO:teuthology.orchestra.run.smithi167.stdout:2 2023-10-05T21:06:17.643 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-05T21:06:17.644 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-05T21:06:17.686 INFO:tasks.ceph.osd.0:Started 2023-10-05T21:06:17.686 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-05T21:06:17.686 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-05T21:06:17.688 INFO:tasks.ceph.osd.1:Started 2023-10-05T21:06:17.689 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-05T21:06:17.689 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-05T21:06:17.691 INFO:tasks.ceph.osd.2:Started 2023-10-05T21:06:17.692 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-05T21:06:17.902 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:06:17.903 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":5,"fsid":"18db5e25-e3f3-4284-9e58-5e5554ae5140","created":"2023-10-05T21:06:16.596517+0000","modified":"2023-10-05T21:06:17.629595+0000","last_up_change":"0.000000","last_in_change":"2023-10-05T21:06:17.629595+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"ab74b673-a624-4ec2-ad02-409b63519337","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"77f471b1-fe86-4b52-8091-01e503bab3c0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2e146899-f47c-4e3e-aeb0-737ab3b287d8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-05T21:06:17.915 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-05T21:06:17.915 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-05T21:06:17.974 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-05T21:06:17.972+0000 7efdae9541c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-05T21:06:18.111 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-05T21:06:18.109+0000 7efdae9541c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-05T21:06:18.216 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-05T21:06:18.295 INFO:tasks.ceph.osd.2.smithi167.stderr:2023-10-05T21:06:18.294+0000 7f54e0755540 -1 Falling back to public interface 2023-10-05T21:06:18.303 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:06:18.301+0000 7f6cc99e9540 -1 Falling back to public interface 2023-10-05T21:06:18.314 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:06:18.312+0000 7ff694188540 -1 Falling back to public interface 2023-10-05T21:06:18.406 INFO:teuthology.misc.health.smithi167.stdout: 2023-10-05T21:06:18.406 INFO:teuthology.misc.health.smithi167.stdout:{"epoch":5,"fsid":"18db5e25-e3f3-4284-9e58-5e5554ae5140","created":"2023-10-05T21:06:16.596517+0000","modified":"2023-10-05T21:06:17.629595+0000","last_up_change":"0.000000","last_in_change":"2023-10-05T21:06:17.629595+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"ab74b673-a624-4ec2-ad02-409b63519337","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"77f471b1-fe86-4b52-8091-01e503bab3c0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2e146899-f47c-4e3e-aeb0-737ab3b287d8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-05T21:06:18.417 DEBUG:teuthology.misc:0 of 3 OSDs are up 2023-10-05T21:06:18.473 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-05T21:06:18.471+0000 7efdae9541c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-05T21:06:19.198 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-05T21:06:19.196+0000 7efdae9541c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-05T21:06:19.268 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-05T21:06:19.267+0000 7efdae9541c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-05T21:06:19.402 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-05T21:06:19.401+0000 7efdae9541c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-05T21:06:19.470 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-05T21:06:19.469+0000 7efdae9541c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-05T21:06:19.650 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-05T21:06:19.648+0000 7efdae9541c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-05T21:06:19.727 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-05T21:06:19.725+0000 7efdae9541c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-05T21:06:19.802 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-05T21:06:19.801+0000 7efdae9541c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-05T21:06:19.899 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-05T21:06:19.897+0000 7efdae9541c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-05T21:06:20.078 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-05T21:06:20.076+0000 7efdae9541c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-05T21:06:20.157 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-05T21:06:20.155+0000 7efdae9541c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-05T21:06:20.445 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-05T21:06:20.443+0000 7efdae9541c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-05T21:06:20.516 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-05T21:06:20.514+0000 7efdae9541c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-05T21:06:21.036 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-05T21:06:21.034+0000 7efdae9541c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-05T21:06:21.404 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-05T21:06:21.402+0000 7efdae9541c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-05T21:06:21.550 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-05T21:06:21.548+0000 7efdae9541c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-05T21:06:21.800 INFO:tasks.ceph.osd.2.smithi167.stderr:2023-10-05T21:06:21.798+0000 7f54e0755540 -1 osd.2 0 log_to_monitors true 2023-10-05T21:06:21.813 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:06:21.811+0000 7f6cc99e9540 -1 osd.1 0 log_to_monitors true 2023-10-05T21:06:21.814 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:06:21.812+0000 7ff694188540 -1 osd.0 0 log_to_monitors true 2023-10-05T21:06:21.917 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-05T21:06:21.915+0000 7efdae9541c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-05T21:06:22.061 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-05T21:06:22.059+0000 7efdae9541c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-05T21:06:22.210 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-05T21:06:22.208+0000 7efdae9541c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-05T21:06:22.310 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-05T21:06:22.308+0000 7efdae9541c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-05T21:06:23.532 INFO:tasks.ceph.mgr.x.smithi167.stderr:2023-10-05T21:06:23.530+0000 7efd73035700 -1 mgr.server handle_report got status from non-daemon mon.a 2023-10-05T21:06:24.719 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-05T21:06:25.048 INFO:teuthology.misc.health.smithi167.stdout: 2023-10-05T21:06:25.048 INFO:teuthology.misc.health.smithi167.stdout:{"epoch":7,"fsid":"18db5e25-e3f3-4284-9e58-5e5554ae5140","created":"2023-10-05T21:06:16.596517+0000","modified":"2023-10-05T21:06:23.573376+0000","last_up_change":"0.000000","last_in_change":"2023-10-05T21:06:17.629595+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"ab74b673-a624-4ec2-ad02-409b63519337","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"77f471b1-fe86-4b52-8091-01e503bab3c0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2e146899-f47c-4e3e-aeb0-737ab3b287d8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-05T21:06:25.061 DEBUG:teuthology.misc:0 of 3 OSDs are up 2023-10-05T21:06:26.613 INFO:tasks.ceph.osd.2.smithi167.stderr:2023-10-05T21:06:26.611+0000 7f54d735a700 -1 osd.2 0 waiting for initial osdmap 2023-10-05T21:06:26.618 INFO:tasks.ceph.osd.2.smithi167.stderr:2023-10-05T21:06:26.616+0000 7f54d0151700 -1 osd.2 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-05T21:06:26.622 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:06:26.620+0000 7ff68ad8d700 -1 osd.0 0 waiting for initial osdmap 2023-10-05T21:06:26.626 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:06:26.624+0000 7ff683b84700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-05T21:06:26.631 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:06:26.629+0000 7f6cc05ee700 -1 osd.1 0 waiting for initial osdmap 2023-10-05T21:06:26.634 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:06:26.632+0000 7f6cb93e5700 -1 osd.1 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-05T21:06:31.364 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-05T21:06:31.679 INFO:teuthology.misc.health.smithi167.stdout: 2023-10-05T21:06:31.679 INFO:teuthology.misc.health.smithi167.stdout:{"epoch":11,"fsid":"18db5e25-e3f3-4284-9e58-5e5554ae5140","created":"2023-10-05T21:06:16.596517+0000","modified":"2023-10-05T21:06:30.621917+0000","last_up_change":"2023-10-05T21:06:27.614053+0000","last_in_change":"2023-10-05T21:06:17.629595+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-05T21:06:28.369471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"ab74b673-a624-4ec2-ad02-409b63519337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":144114},{"type":"v1","addr":"172.21.15.167:6817","nonce":144114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":144114},{"type":"v1","addr":"172.21.15.167:6819","nonce":144114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":144114},{"type":"v1","addr":"172.21.15.167:6823","nonce":144114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":144114},{"type":"v1","addr":"172.21.15.167:6821","nonce":144114}]},"public_addr":"172.21.15.167:6817/144114","cluster_addr":"172.21.15.167:6819/144114","heartbeat_back_addr":"172.21.15.167:6823/144114","heartbeat_front_addr":"172.21.15.167:6821/144114","state":["exists","up"]},{"osd":1,"uuid":"77f471b1-fe86-4b52-8091-01e503bab3c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":144115},{"type":"v1","addr":"172.21.15.167:6809","nonce":144115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":144115},{"type":"v1","addr":"172.21.15.167:6811","nonce":144115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":144115},{"type":"v1","addr":"172.21.15.167:6815","nonce":144115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":144115},{"type":"v1","addr":"172.21.15.167:6813","nonce":144115}]},"public_addr":"172.21.15.167:6809/144115","cluster_addr":"172.21.15.167:6811/144115","heartbeat_back_addr":"172.21.15.167:6815/144115","heartbeat_front_addr":"172.21.15.167:6813/144115","state":["exists","up"]},{"osd":2,"uuid":"2e146899-f47c-4e3e-aeb0-737ab3b287d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":144105},{"type":"v1","addr":"172.21.15.167:6801","nonce":144105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":144105},{"type":"v1","addr":"172.21.15.167:6803","nonce":144105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":144105},{"type":"v1","addr":"172.21.15.167:6807","nonce":144105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":144105},{"type":"v1","addr":"172.21.15.167:6805","nonce":144105}]},"public_addr":"172.21.15.167:6801/144105","cluster_addr":"172.21.15.167:6803/144105","heartbeat_back_addr":"172.21.15.167:6807/144105","heartbeat_front_addr":"172.21.15.167:6805/144105","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-05T21:06:22.818703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-05T21:06:22.854292+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-05T21:06:22.794012+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-05T21:06:31.691 DEBUG:teuthology.misc:3 of 3 OSDs are up 2023-10-05T21:06:31.692 INFO:tasks.ceph:Creating RBD pool 2023-10-05T21:06:31.692 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd pool create rbd 8 2023-10-05T21:06:32.642 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'rbd' created 2023-10-05T21:06:32.655 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-10-05T21:06:33.650 INFO:teuthology.orchestra.run.smithi167.stderr:enabled application 'rbd' on pool 'rbd' 2023-10-05T21:06:33.664 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-05T21:06:33.665 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-05T21:06:33.665 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-05T21:06:33.665 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-05T21:06:33.666 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-05T21:06:34.033 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:06:34.045 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":5,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":143693},{"type":"v1","addr":"172.21.15.167:6825","nonce":143693}]},"active_addr":"172.21.15.167:6825/143693","active_change":"2023-10-05T21:06:22.323236+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.167:0","nonce":3307253145}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.167:0","nonce":468993766}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.167:0","nonce":2591400608}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.167:0","nonce":567029105}]}]} 2023-10-05T21:06:34.048 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-05T21:06:34.048 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-05T21:06:34.049 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-05T21:06:34.358 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:06:34.358 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":13,"fsid":"18db5e25-e3f3-4284-9e58-5e5554ae5140","created":"2023-10-05T21:06:16.596517+0000","modified":"2023-10-05T21:06:33.631047+0000","last_up_change":"2023-10-05T21:06:27.614053+0000","last_in_change":"2023-10-05T21:06:17.629595+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-05T21:06:28.369471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-05T21:06:32.019894+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"ab74b673-a624-4ec2-ad02-409b63519337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":144114},{"type":"v1","addr":"172.21.15.167:6817","nonce":144114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":144114},{"type":"v1","addr":"172.21.15.167:6819","nonce":144114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":144114},{"type":"v1","addr":"172.21.15.167:6823","nonce":144114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":144114},{"type":"v1","addr":"172.21.15.167:6821","nonce":144114}]},"public_addr":"172.21.15.167:6817/144114","cluster_addr":"172.21.15.167:6819/144114","heartbeat_back_addr":"172.21.15.167:6823/144114","heartbeat_front_addr":"172.21.15.167:6821/144114","state":["exists","up"]},{"osd":1,"uuid":"77f471b1-fe86-4b52-8091-01e503bab3c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":144115},{"type":"v1","addr":"172.21.15.167:6809","nonce":144115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":144115},{"type":"v1","addr":"172.21.15.167:6811","nonce":144115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":144115},{"type":"v1","addr":"172.21.15.167:6815","nonce":144115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":144115},{"type":"v1","addr":"172.21.15.167:6813","nonce":144115}]},"public_addr":"172.21.15.167:6809/144115","cluster_addr":"172.21.15.167:6811/144115","heartbeat_back_addr":"172.21.15.167:6815/144115","heartbeat_front_addr":"172.21.15.167:6813/144115","state":["exists","up"]},{"osd":2,"uuid":"2e146899-f47c-4e3e-aeb0-737ab3b287d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":144105},{"type":"v1","addr":"172.21.15.167:6801","nonce":144105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":144105},{"type":"v1","addr":"172.21.15.167:6803","nonce":144105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":144105},{"type":"v1","addr":"172.21.15.167:6807","nonce":144105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":144105},{"type":"v1","addr":"172.21.15.167:6805","nonce":144105}]},"public_addr":"172.21.15.167:6801/144105","cluster_addr":"172.21.15.167:6803/144105","heartbeat_back_addr":"172.21.15.167:6807/144105","heartbeat_front_addr":"172.21.15.167:6805/144105","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-05T21:06:22.818703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-05T21:06:22.854292+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-05T21:06:22.794012+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-05T21:06:34.372 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-05T21:06:34.372 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-05T21:06:34.682 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:06:34.682 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":14,"fsid":"18db5e25-e3f3-4284-9e58-5e5554ae5140","created":"2023-10-05T21:06:16.596517+0000","modified":"2023-10-05T21:06:34.639617+0000","last_up_change":"2023-10-05T21:06:27.614053+0000","last_in_change":"2023-10-05T21:06:17.629595+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-05T21:06:28.369471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-05T21:06:32.019894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"ab74b673-a624-4ec2-ad02-409b63519337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":144114},{"type":"v1","addr":"172.21.15.167:6817","nonce":144114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":144114},{"type":"v1","addr":"172.21.15.167:6819","nonce":144114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":144114},{"type":"v1","addr":"172.21.15.167:6823","nonce":144114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":144114},{"type":"v1","addr":"172.21.15.167:6821","nonce":144114}]},"public_addr":"172.21.15.167:6817/144114","cluster_addr":"172.21.15.167:6819/144114","heartbeat_back_addr":"172.21.15.167:6823/144114","heartbeat_front_addr":"172.21.15.167:6821/144114","state":["exists","up"]},{"osd":1,"uuid":"77f471b1-fe86-4b52-8091-01e503bab3c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":144115},{"type":"v1","addr":"172.21.15.167:6809","nonce":144115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":144115},{"type":"v1","addr":"172.21.15.167:6811","nonce":144115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":144115},{"type":"v1","addr":"172.21.15.167:6815","nonce":144115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":144115},{"type":"v1","addr":"172.21.15.167:6813","nonce":144115}]},"public_addr":"172.21.15.167:6809/144115","cluster_addr":"172.21.15.167:6811/144115","heartbeat_back_addr":"172.21.15.167:6815/144115","heartbeat_front_addr":"172.21.15.167:6813/144115","state":["exists","up"]},{"osd":2,"uuid":"2e146899-f47c-4e3e-aeb0-737ab3b287d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":144105},{"type":"v1","addr":"172.21.15.167:6801","nonce":144105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":144105},{"type":"v1","addr":"172.21.15.167:6803","nonce":144105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":144105},{"type":"v1","addr":"172.21.15.167:6807","nonce":144105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":144105},{"type":"v1","addr":"172.21.15.167:6805","nonce":144105}]},"public_addr":"172.21.15.167:6801/144105","cluster_addr":"172.21.15.167:6803/144105","heartbeat_back_addr":"172.21.15.167:6807/144105","heartbeat_front_addr":"172.21.15.167:6805/144105","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-05T21:06:22.818703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-05T21:06:22.854292+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-05T21:06:22.794012+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-05T21:06:34.696 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-05T21:06:34.696 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-05T21:06:34.697 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-05T21:06:34.857 INFO:teuthology.orchestra.run.smithi167.stdout:34359738373 2023-10-05T21:06:34.858 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-05T21:06:34.861 INFO:teuthology.orchestra.run.smithi167.stdout:34359738372 2023-10-05T21:06:34.861 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-05T21:06:34.863 INFO:teuthology.orchestra.run.smithi167.stdout:34359738374 2023-10-05T21:06:34.863 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-05T21:06:35.204 INFO:teuthology.orchestra.run.smithi167.stdout:34359738372 2023-10-05T21:06:35.218 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738372 for osd.2 2023-10-05T21:06:35.232 INFO:teuthology.orchestra.run.smithi167.stdout:34359738373 2023-10-05T21:06:35.246 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738374 got 34359738373 for osd.0 2023-10-05T21:06:35.249 INFO:teuthology.orchestra.run.smithi167.stdout:34359738371 2023-10-05T21:06:35.263 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738372 got 34359738371 for osd.1 2023-10-05T21:06:36.219 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-05T21:06:36.247 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-05T21:06:36.264 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-05T21:06:36.562 INFO:teuthology.orchestra.run.smithi167.stdout:34359738372 2023-10-05T21:06:36.576 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738372 for osd.2 2023-10-05T21:06:36.596 INFO:teuthology.orchestra.run.smithi167.stdout:34359738371 2023-10-05T21:06:36.602 INFO:teuthology.orchestra.run.smithi167.stdout:34359738373 2023-10-05T21:06:36.611 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738372 got 34359738371 for osd.1 2023-10-05T21:06:36.616 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738374 got 34359738373 for osd.0 2023-10-05T21:06:37.577 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-05T21:06:37.612 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-05T21:06:37.617 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-05T21:06:37.919 INFO:teuthology.orchestra.run.smithi167.stdout:34359738373 2023-10-05T21:06:37.933 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738373 for osd.2 2023-10-05T21:06:37.933 DEBUG:teuthology.parallel:result is None 2023-10-05T21:06:37.952 INFO:teuthology.orchestra.run.smithi167.stdout:34359738374 2023-10-05T21:06:37.954 INFO:teuthology.orchestra.run.smithi167.stdout:34359738372 2023-10-05T21:06:37.967 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738374 got 34359738374 for osd.0 2023-10-05T21:06:37.967 DEBUG:teuthology.parallel:result is None 2023-10-05T21:06:37.969 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738372 got 34359738372 for osd.1 2023-10-05T21:06:37.969 DEBUG:teuthology.parallel:result is None 2023-10-05T21:06:37.969 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-05T21:06:37.970 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-05T21:06:38.314 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:06:38.316 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-05T21:06:38.327 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2023-10-05T21:06:36.328540+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":892312,"kb_used_data":448,"kb_used_omap":0,"kb_used_meta":891840,"kb_avail":282223208,"statfs":{"total":289910292480,"available":288996564992,"internally_reserved":0,"allocated":458752,"data_stored":1274213,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":913244160},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.686270"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-10-05T21:06:33.652966+0000","last_change":"2023-10-05T21:06:33.648875+0000","last_active":"2023-10-05T21:06:33.652966+0000","last_peered":"2023-10-05T21:06:33.652966+0000","last_clean":"2023-10-05T21:06:33.652966+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-05T21:06:33.652966+0000","last_undegraded":"2023-10-05T21:06:33.652966+0000","last_fullsized":"2023-10-05T21:06:33.652966+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T00:45:46.698848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-10-05T21:06:33.652934+0000","last_change":"2023-10-05T21:06:33.648684+0000","last_active":"2023-10-05T21:06:33.652934+0000","last_peered":"2023-10-05T21:06:33.652934+0000","last_clean":"2023-10-05T21:06:33.652934+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-05T21:06:33.652934+0000","last_undegraded":"2023-10-05T21:06:33.652934+0000","last_fullsized":"2023-10-05T21:06:33.652934+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:22.382648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-10-05T21:06:33.655595+0000","last_change":"2023-10-05T21:06:33.651184+0000","last_active":"2023-10-05T21:06:33.655595+0000","last_peered":"2023-10-05T21:06:33.655595+0000","last_clean":"2023-10-05T21:06:33.655595+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-05T21:06:33.655595+0000","last_undegraded":"2023-10-05T21:06:33.655595+0000","last_fullsized":"2023-10-05T21:06:33.655595+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:17.807335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-10-05T21:06:33.651047+0000","last_change":"2023-10-05T21:06:33.651047+0000","last_active":"2023-10-05T21:06:33.651047+0000","last_peered":"2023-10-05T21:06:33.651047+0000","last_clean":"2023-10-05T21:06:33.651047+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-05T21:06:33.651047+0000","last_undegraded":"2023-10-05T21:06:33.651047+0000","last_fullsized":"2023-10-05T21:06:33.651047+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:52.571985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":139,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-10-05T21:06:33.645705+0000","last_change":"2023-10-05T21:06:29.922199+0000","last_active":"2023-10-05T21:06:33.645705+0000","last_peered":"2023-10-05T21:06:33.645705+0000","last_clean":"2023-10-05T21:06:33.645705+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-05T21:06:33.645705+0000","last_undegraded":"2023-10-05T21:06:33.645705+0000","last_fullsized":"2023-10-05T21:06:33.645705+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:28.614917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:28.614917+0000","last_clean_scrub_stamp":"2023-10-05T21:06:28.614917+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:59:56.310688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-10-05T21:06:33.653076+0000","last_change":"2023-10-05T21:06:33.649744+0000","last_active":"2023-10-05T21:06:33.653076+0000","last_peered":"2023-10-05T21:06:33.653076+0000","last_clean":"2023-10-05T21:06:33.653076+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-05T21:06:33.653076+0000","last_undegraded":"2023-10-05T21:06:33.653076+0000","last_fullsized":"2023-10-05T21:06:33.653076+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-06T22:30:43.667801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-10-05T21:06:33.651180+0000","last_change":"2023-10-05T21:06:33.651180+0000","last_active":"2023-10-05T21:06:33.651180+0000","last_peered":"2023-10-05T21:06:33.651180+0000","last_clean":"2023-10-05T21:06:33.651180+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-05T21:06:33.651180+0000","last_undegraded":"2023-10-05T21:06:33.651180+0000","last_fullsized":"2023-10-05T21:06:33.651180+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:20:06.851803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-10-05T21:06:33.649007+0000","last_change":"2023-10-05T21:06:33.649007+0000","last_active":"2023-10-05T21:06:33.649007+0000","last_peered":"2023-10-05T21:06:33.649007+0000","last_clean":"2023-10-05T21:06:33.649007+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-05T21:06:33.649007+0000","last_undegraded":"2023-10-05T21:06:33.649007+0000","last_fullsized":"2023-10-05T21:06:33.649007+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T01:49:48.848559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-10-05T21:06:33.655648+0000","last_change":"2023-10-05T21:06:33.651381+0000","last_active":"2023-10-05T21:06:33.655648+0000","last_peered":"2023-10-05T21:06:33.655648+0000","last_clean":"2023-10-05T21:06:33.655648+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-05T21:06:33.655648+0000","last_undegraded":"2023-10-05T21:06:33.655648+0000","last_fullsized":"2023-10-05T21:06:33.655648+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T01:16:04.809466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738373,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297400,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074440,"statfs":{"total":96636764160,"available":96332226560,"internally_reserved":0,"allocated":114688,"data_stored":31159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738374,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297456,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074384,"statfs":{"total":96636764160,"available":96332169216,"internally_reserved":0,"allocated":172032,"data_stored":621527,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]}]},{"osd":1,"up_from":8,"seq":34359738372,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297456,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074384,"statfs":{"total":96636764160,"available":96332169216,"internally_reserved":0,"allocated":172032,"data_stored":621527,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-05T21:06:38.328 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-05T21:06:38.633 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:06:38.635 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-05T21:06:38.646 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2023-10-05T21:06:38.328876+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":892312,"kb_used_data":448,"kb_used_omap":0,"kb_used_meta":891840,"kb_avail":282223208,"statfs":{"total":289910292480,"available":288996564992,"internally_reserved":0,"allocated":458752,"data_stored":1274213,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":913244160},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.686607"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-10-05T21:06:33.652966+0000","last_change":"2023-10-05T21:06:33.648875+0000","last_active":"2023-10-05T21:06:33.652966+0000","last_peered":"2023-10-05T21:06:33.652966+0000","last_clean":"2023-10-05T21:06:33.652966+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-05T21:06:33.652966+0000","last_undegraded":"2023-10-05T21:06:33.652966+0000","last_fullsized":"2023-10-05T21:06:33.652966+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T00:45:46.698848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-10-05T21:06:33.652934+0000","last_change":"2023-10-05T21:06:33.648684+0000","last_active":"2023-10-05T21:06:33.652934+0000","last_peered":"2023-10-05T21:06:33.652934+0000","last_clean":"2023-10-05T21:06:33.652934+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-05T21:06:33.652934+0000","last_undegraded":"2023-10-05T21:06:33.652934+0000","last_fullsized":"2023-10-05T21:06:33.652934+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:22.382648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-10-05T21:06:33.655595+0000","last_change":"2023-10-05T21:06:33.651184+0000","last_active":"2023-10-05T21:06:33.655595+0000","last_peered":"2023-10-05T21:06:33.655595+0000","last_clean":"2023-10-05T21:06:33.655595+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-05T21:06:33.655595+0000","last_undegraded":"2023-10-05T21:06:33.655595+0000","last_fullsized":"2023-10-05T21:06:33.655595+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:17.807335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-10-05T21:06:33.651047+0000","last_change":"2023-10-05T21:06:33.651047+0000","last_active":"2023-10-05T21:06:33.651047+0000","last_peered":"2023-10-05T21:06:33.651047+0000","last_clean":"2023-10-05T21:06:33.651047+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-05T21:06:33.651047+0000","last_undegraded":"2023-10-05T21:06:33.651047+0000","last_fullsized":"2023-10-05T21:06:33.651047+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:52.571985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":139,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-10-05T21:06:33.645705+0000","last_change":"2023-10-05T21:06:29.922199+0000","last_active":"2023-10-05T21:06:33.645705+0000","last_peered":"2023-10-05T21:06:33.645705+0000","last_clean":"2023-10-05T21:06:33.645705+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-05T21:06:33.645705+0000","last_undegraded":"2023-10-05T21:06:33.645705+0000","last_fullsized":"2023-10-05T21:06:33.645705+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:28.614917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:28.614917+0000","last_clean_scrub_stamp":"2023-10-05T21:06:28.614917+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:59:56.310688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-10-05T21:06:33.653076+0000","last_change":"2023-10-05T21:06:33.649744+0000","last_active":"2023-10-05T21:06:33.653076+0000","last_peered":"2023-10-05T21:06:33.653076+0000","last_clean":"2023-10-05T21:06:33.653076+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-05T21:06:33.653076+0000","last_undegraded":"2023-10-05T21:06:33.653076+0000","last_fullsized":"2023-10-05T21:06:33.653076+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-06T22:30:43.667801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-10-05T21:06:33.651180+0000","last_change":"2023-10-05T21:06:33.651180+0000","last_active":"2023-10-05T21:06:33.651180+0000","last_peered":"2023-10-05T21:06:33.651180+0000","last_clean":"2023-10-05T21:06:33.651180+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-05T21:06:33.651180+0000","last_undegraded":"2023-10-05T21:06:33.651180+0000","last_fullsized":"2023-10-05T21:06:33.651180+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:20:06.851803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-10-05T21:06:33.649007+0000","last_change":"2023-10-05T21:06:33.649007+0000","last_active":"2023-10-05T21:06:33.649007+0000","last_peered":"2023-10-05T21:06:33.649007+0000","last_clean":"2023-10-05T21:06:33.649007+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-05T21:06:33.649007+0000","last_undegraded":"2023-10-05T21:06:33.649007+0000","last_fullsized":"2023-10-05T21:06:33.649007+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T01:49:48.848559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-10-05T21:06:33.655648+0000","last_change":"2023-10-05T21:06:33.651381+0000","last_active":"2023-10-05T21:06:33.655648+0000","last_peered":"2023-10-05T21:06:33.655648+0000","last_clean":"2023-10-05T21:06:33.655648+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-05T21:06:33.655648+0000","last_undegraded":"2023-10-05T21:06:33.655648+0000","last_fullsized":"2023-10-05T21:06:33.655648+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T01:16:04.809466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738373,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297400,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074440,"statfs":{"total":96636764160,"available":96332226560,"internally_reserved":0,"allocated":114688,"data_stored":31159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738374,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297456,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074384,"statfs":{"total":96636764160,"available":96332169216,"internally_reserved":0,"allocated":172032,"data_stored":621527,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]}]},{"osd":1,"up_from":8,"seq":34359738372,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297456,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074384,"statfs":{"total":96636764160,"available":96332169216,"internally_reserved":0,"allocated":172032,"data_stored":621527,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-05T21:06:38.647 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-05T21:06:38.647 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-05T21:06:38.647 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-05T21:06:38.647 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-05T21:06:38.993 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-05T21:06:38.993 INFO:teuthology.orchestra.run.smithi167.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-05T21:06:39.007 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-05T21:06:39.007 INFO:teuthology.run_tasks:Running task exec... 2023-10-05T21:06:39.016 INFO:teuthology.task.exec:Executing custom commands... 2023-10-05T21:06:39.017 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi167.front.sepia.ceph.com 2023-10-05T21:06:39.017 DEBUG:teuthology.orchestra.run.smithi167:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool create cache 4' 2023-10-05T21:06:39.659 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cache' created 2023-10-05T21:06:39.671 DEBUG:teuthology.orchestra.run.smithi167:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd tier add rbd cache' 2023-10-05T21:06:40.666 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cache' is now (or already was) a tier of 'rbd' 2023-10-05T21:06:40.680 DEBUG:teuthology.orchestra.run.smithi167:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd tier cache-mode cache writeback' 2023-10-05T21:06:41.665 INFO:teuthology.orchestra.run.smithi167.stderr:set cache-mode for pool 'cache' to writeback 2023-10-05T21:06:41.680 DEBUG:teuthology.orchestra.run.smithi167:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd tier set-overlay rbd cache' 2023-10-05T21:06:42.671 INFO:teuthology.orchestra.run.smithi167.stderr:overlay for 'rbd' is now (or already was) 'cache' 2023-10-05T21:06:42.684 DEBUG:teuthology.orchestra.run.smithi167:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache hit_set_type bloom' 2023-10-05T21:06:43.673 INFO:teuthology.orchestra.run.smithi167.stderr:set pool 3 hit_set_type to bloom 2023-10-05T21:06:43.687 DEBUG:teuthology.orchestra.run.smithi167:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache hit_set_count 8' 2023-10-05T21:06:44.685 INFO:teuthology.orchestra.run.smithi167.stderr:set pool 3 hit_set_count to 8 2023-10-05T21:06:44.699 DEBUG:teuthology.orchestra.run.smithi167:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache hit_set_period 60' 2023-10-05T21:06:45.686 INFO:teuthology.orchestra.run.smithi167.stderr:set pool 3 hit_set_period to 60 2023-10-05T21:06:45.701 DEBUG:teuthology.orchestra.run.smithi167:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache target_max_objects 250' 2023-10-05T21:06:46.703 INFO:teuthology.orchestra.run.smithi167.stderr:set pool 3 target_max_objects to 250 2023-10-05T21:06:46.717 INFO:teuthology.run_tasks:Running task workunit... 2023-10-05T21:06:46.735 INFO:tasks.workunit:Pulling workunits from ref 487ffcb5ddb60d29c5688d1a457e812b1b697b90 2023-10-05T21:06:46.735 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-10-05T21:06:46.735 DEBUG:teuthology.orchestra.run.smithi167:> stat -- /home/ubuntu/cephtest/mnt.0 2023-10-05T21:06:46.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-05T21:06:46.755 INFO:teuthology.orchestra.run.smithi167.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-05T21:06:46.755 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -- /home/ubuntu/cephtest/mnt.0 2023-10-05T21:06:46.815 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2023-10-05T21:06:46.815 DEBUG:teuthology.orchestra.run.smithi167:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2023-10-05T21:06:46.873 INFO:tasks.workunit:timeout=3h 2023-10-05T21:06:46.874 INFO:tasks.workunit:cleanup=True 2023-10-05T21:06:46.874 DEBUG:teuthology.orchestra.run.smithi167:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 487ffcb5ddb60d29c5688d1a457e812b1b697b90 2023-10-05T21:06:46.933 INFO:tasks.workunit.client.0.smithi167.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-10-05T21:07:29.287 INFO:tasks.workunit.client.0.smithi167.stderr:Note: switching to '487ffcb5ddb60d29c5688d1a457e812b1b697b90'. 2023-10-05T21:07:29.287 INFO:tasks.workunit.client.0.smithi167.stderr: 2023-10-05T21:07:29.287 INFO:tasks.workunit.client.0.smithi167.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-10-05T21:07:29.287 INFO:tasks.workunit.client.0.smithi167.stderr:changes and commit them, and you can discard any commits you make in this 2023-10-05T21:07:29.287 INFO:tasks.workunit.client.0.smithi167.stderr:state without impacting any branches by switching back to a branch. 2023-10-05T21:07:29.287 INFO:tasks.workunit.client.0.smithi167.stderr: 2023-10-05T21:07:29.288 INFO:tasks.workunit.client.0.smithi167.stderr:If you want to create a new branch to retain commits you create, you may 2023-10-05T21:07:29.288 INFO:tasks.workunit.client.0.smithi167.stderr:do so (now or later) by using -c with the switch command. Example: 2023-10-05T21:07:29.288 INFO:tasks.workunit.client.0.smithi167.stderr: 2023-10-05T21:07:29.288 INFO:tasks.workunit.client.0.smithi167.stderr: git switch -c 2023-10-05T21:07:29.288 INFO:tasks.workunit.client.0.smithi167.stderr: 2023-10-05T21:07:29.288 INFO:tasks.workunit.client.0.smithi167.stderr:Or undo this operation with: 2023-10-05T21:07:29.288 INFO:tasks.workunit.client.0.smithi167.stderr: 2023-10-05T21:07:29.288 INFO:tasks.workunit.client.0.smithi167.stderr: git switch - 2023-10-05T21:07:29.288 INFO:tasks.workunit.client.0.smithi167.stderr: 2023-10-05T21:07:29.288 INFO:tasks.workunit.client.0.smithi167.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-10-05T21:07:29.288 INFO:tasks.workunit.client.0.smithi167.stderr: 2023-10-05T21:07:29.288 INFO:tasks.workunit.client.0.smithi167.stderr:HEAD is now at 487ffcb5ddb Merge branch 'wip-62943-quincy' of https://github.com/cfsnyder/ceph into wip-yuri2-testing-2023-10-04-1333-quincy 2023-10-05T21:07:29.292 DEBUG:teuthology.orchestra.run.smithi167:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-10-05T21:07:29.362 INFO:tasks.workunit.client.0.smithi167.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-10-05T21:07:29.364 INFO:tasks.workunit.client.0.smithi167.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-10-05T21:07:29.364 INFO:tasks.workunit.client.0.smithi167.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-10-05T21:07:29.934 INFO:tasks.workunit.client.0.smithi167.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-10-05T21:07:29.990 INFO:tasks.workunit.client.0.smithi167.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-10-05T21:07:30.028 INFO:tasks.workunit.client.0.smithi167.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-10-05T21:07:30.029 INFO:tasks.workunit.client.0.smithi167.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-10-05T21:07:30.029 INFO:tasks.workunit.client.0.smithi167.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-10-05T21:07:30.066 INFO:tasks.workunit.client.0.smithi167.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-10-05T21:07:30.069 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2023-10-05T21:07:30.069 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-10-05T21:07:30.128 INFO:tasks.workunit:Running workunits matching rbd/cli_generic.sh on client.0... 2023-10-05T21:07:30.129 INFO:tasks.workunit:Running workunit rbd/cli_generic.sh... 2023-10-05T21:07:30.129 DEBUG:teuthology.orchestra.run.smithi167:workunit test rbd/cli_generic.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=487ffcb5ddb60d29c5688d1a457e812b1b697b90 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/cli_generic.sh 2023-10-05T21:07:30.190 INFO:tasks.workunit.client.0.smithi167.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/cli_generic.sh 2023-10-05T21:07:30.191 INFO:tasks.workunit.client.0.smithi167.stderr:+ . /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/../../standalone/ceph-helpers.sh 2023-10-05T21:07:30.191 INFO:tasks.workunit.client.0.smithi167.stderr:++ TIMEOUT=300 2023-10-05T21:07:30.192 INFO:tasks.workunit.client.0.smithi167.stderr:++ WAIT_FOR_CLEAN_TIMEOUT=90 2023-10-05T21:07:30.192 INFO:tasks.workunit.client.0.smithi167.stderr:++ MAX_TIMEOUT=15 2023-10-05T21:07:30.192 INFO:tasks.workunit.client.0.smithi167.stderr:++ PG_NUM=4 2023-10-05T21:07:30.192 INFO:tasks.workunit.client.0.smithi167.stderr:++ TMPDIR=/tmp 2023-10-05T21:07:30.192 INFO:tasks.workunit.client.0.smithi167.stderr:++ CEPH_BUILD_VIRTUALENV=/tmp 2023-10-05T21:07:30.192 INFO:tasks.workunit.client.0.smithi167.stderr:++ TESTDIR=/home/ubuntu/cephtest 2023-10-05T21:07:30.192 INFO:tasks.workunit.client.0.smithi167.stderr:++ type xmlstarlet 2023-10-05T21:07:30.192 INFO:tasks.workunit.client.0.smithi167.stderr:++ XMLSTARLET=xmlstarlet 2023-10-05T21:07:30.192 INFO:tasks.workunit.client.0.smithi167.stderr:+++ uname 2023-10-05T21:07:30.194 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' Linux = FreeBSD ']' 2023-10-05T21:07:30.194 INFO:tasks.workunit.client.0.smithi167.stderr:++ SED=sed 2023-10-05T21:07:30.194 INFO:tasks.workunit.client.0.smithi167.stderr:++ AWK=awk 2023-10-05T21:07:30.194 INFO:tasks.workunit.client.0.smithi167.stderr:+++ stty -a 2023-10-05T21:07:30.194 INFO:tasks.workunit.client.0.smithi167.stderr:+++ head -1 2023-10-05T21:07:30.194 INFO:tasks.workunit.client.0.smithi167.stderr:+++ sed -e 's/.*columns \([0-9]*\).*/\1/' 2023-10-05T21:07:30.194 INFO:tasks.workunit.client.0.smithi167.stderr:stty: 'standard input': Inappropriate ioctl for device 2023-10-05T21:07:30.194 INFO:tasks.workunit.client.0.smithi167.stderr:++ termwidth= 2023-10-05T21:07:30.194 INFO:tasks.workunit.client.0.smithi167.stderr:++ '[' -n '' -a '' '!=' 0 ']' 2023-10-05T21:07:30.194 INFO:tasks.workunit.client.0.smithi167.stderr:++ DIFFCOLOPTS='-y ' 2023-10-05T21:07:30.195 INFO:tasks.workunit.client.0.smithi167.stderr:++ KERNCORE=kernel.core_pattern 2023-10-05T21:07:30.195 INFO:tasks.workunit.client.0.smithi167.stderr:++ EXTRA_OPTS= 2023-10-05T21:07:30.199 INFO:tasks.workunit.client.0.smithi167.stderr:++ test '' = TESTS 2023-10-05T21:07:30.199 INFO:tasks.workunit.client.0.smithi167.stderr:+ export RBD_FORCE_ALLOW_V1=1 2023-10-05T21:07:30.199 INFO:tasks.workunit.client.0.smithi167.stderr:+ RBD_FORCE_ALLOW_V1=1 2023-10-05T21:07:30.200 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls 2023-10-05T21:07:30.200 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:07:30.200 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep -v '^0$' 2023-10-05T21:07:30.234 INFO:tasks.workunit.client.0.smithi167.stderr:+ IMGS='testimg1 testimg2 testimg3 testimg4 testimg5 testimg6 testimg-diff1 testimg-diff2 testimg-diff3 foo foo2 bar bar2 test1 test2 test3 test4 clone2' 2023-10-05T21:07:30.234 INFO:tasks.workunit.client.0.smithi167.stderr:+ tiered=0 2023-10-05T21:07:30.235 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph osd dump 2023-10-05T21:07:30.235 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '^pool' 2023-10-05T21:07:30.235 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep ''\''rbd'\''' 2023-10-05T21:07:30.235 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep tier 2023-10-05T21:07:30.650 INFO:tasks.workunit.client.0.smithi167.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 18 lfor 18/18/18 flags hashpspool tiers 3 read_tier 3 write_tier 3 stripe_width 0 application rbd 2023-10-05T21:07:30.650 INFO:tasks.workunit.client.0.smithi167.stderr:+ tiered=1 2023-10-05T21:07:30.656 INFO:tasks.workunit.client.0.smithi167.stderr:+ test_pool_image_args 2023-10-05T21:07:30.656 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'testing pool and image args...' 2023-10-05T21:07:30.657 INFO:tasks.workunit.client.0.smithi167.stdout:testing pool and image args... 2023-10-05T21:07:30.657 INFO:tasks.workunit.client.0.smithi167.stderr:+ remove_images 2023-10-05T21:07:30.657 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:30.737 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:30.817 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:30.896 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:30.971 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:31.045 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:31.114 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:31.184 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:31.254 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:31.324 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:31.402 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:31.476 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:31.549 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:31.626 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:31.696 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:31.967 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:32.037 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:32.106 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:32.180 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph osd pool delete test test --yes-i-really-really-mean-it 2023-10-05T21:07:32.592 INFO:tasks.workunit.client.0.smithi167.stderr:pool 'test' does not exist 2023-10-05T21:07:32.601 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph osd pool create test 32 2023-10-05T21:07:33.877 INFO:tasks.workunit.client.0.smithi167.stderr:pool 'test' already exists 2023-10-05T21:07:33.886 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd pool init test 2023-10-05T21:07:36.767 INFO:tasks.workunit.client.0.smithi167.stderr:+ truncate -s 1 /tmp/empty /tmp/empty@snap 2023-10-05T21:07:36.769 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls 2023-10-05T21:07:36.769 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:07:36.769 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 0 2023-10-05T21:07:36.802 INFO:tasks.workunit.client.0.smithi167.stdout:0 2023-10-05T21:07:36.803 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create -s 1 test1 2023-10-05T21:07:36.832 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:36.831+0000 7f4ffbaab540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:38.781 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls 2023-10-05T21:07:38.781 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep -q test1 2023-10-05T21:07:38.812 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd import --image test2 /tmp/empty 2023-10-05T21:07:38.841 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:38.840+0000 7f41b402b540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:38.850 INFO:tasks.workunit.client.0.smithi167.stderr: Importing image: 100% complete...done. 2023-10-05T21:07:38.850 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: --image is deprecated, use --dest 2023-10-05T21:07:38.853 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls 2023-10-05T21:07:38.853 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep -q test2 2023-10-05T21:07:38.881 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd --dest test3 import /tmp/empty 2023-10-05T21:07:38.909 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:38.908+0000 7f2b69888540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:38.920 INFO:tasks.workunit.client.0.smithi167.stderr: Importing image: 100% complete...done. 2023-10-05T21:07:38.923 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls 2023-10-05T21:07:38.923 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep -q test3 2023-10-05T21:07:38.950 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd import /tmp/empty foo 2023-10-05T21:07:38.978 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:38.977+0000 7f675ae7d540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:38.989 INFO:tasks.workunit.client.0.smithi167.stderr: Importing image: 100% complete...done. 2023-10-05T21:07:38.992 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls 2023-10-05T21:07:38.992 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep -q foo 2023-10-05T21:07:39.020 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd import --dest test/empty@snap /tmp/empty 2023-10-05T21:07:39.041 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: destination snapshot name specified for a command that doesn't use it 2023-10-05T21:07:39.042 INFO:tasks.workunit.client.0.smithi167.stderr:+ true 2023-10-05T21:07:39.043 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd import /tmp/empty test/empty@snap 2023-10-05T21:07:39.063 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: destination snapshot name specified for a command that doesn't use it 2023-10-05T21:07:39.064 INFO:tasks.workunit.client.0.smithi167.stderr:+ true 2023-10-05T21:07:39.064 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd import --image test/empty@snap /tmp/empty 2023-10-05T21:07:39.085 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: destination snapshot name specified for a command that doesn't use it 2023-10-05T21:07:39.086 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: --image is deprecated, use --dest 2023-10-05T21:07:39.087 INFO:tasks.workunit.client.0.smithi167.stderr:+ true 2023-10-05T21:07:39.087 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd import /tmp/empty@snap 2023-10-05T21:07:39.106 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: destination snapshot name specified for a command that doesn't use it 2023-10-05T21:07:39.108 INFO:tasks.workunit.client.0.smithi167.stderr:+ true 2023-10-05T21:07:39.108 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls test 2023-10-05T21:07:39.108 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:07:39.108 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 0 2023-10-05T21:07:39.134 INFO:tasks.workunit.client.0.smithi167.stdout:0 2023-10-05T21:07:39.135 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd import /tmp/empty test/test1 2023-10-05T21:07:39.161 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:39.160+0000 7f8731d16540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:40.788 INFO:tasks.workunit.client.0.smithi167.stderr: Importing image: 100% complete...done. 2023-10-05T21:07:40.791 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls test 2023-10-05T21:07:40.791 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep -q test1 2023-10-05T21:07:40.823 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd -p test import /tmp/empty test2 2023-10-05T21:07:40.853 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:40.852+0000 7f55e2af7540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:40.866 INFO:tasks.workunit.client.0.smithi167.stderr: Importing image: 100% complete...done. 2023-10-05T21:07:40.866 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: -p [ --pool ] is deprecated, use --dest-pool 2023-10-05T21:07:40.869 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls test 2023-10-05T21:07:40.869 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep -q test2 2023-10-05T21:07:40.898 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd --image test3 -p test import /tmp/empty 2023-10-05T21:07:40.929 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:40.928+0000 7fb0709bb540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:40.940 INFO:tasks.workunit.client.0.smithi167.stderr: Importing image: 100% complete...done. 2023-10-05T21:07:40.940 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: --image is deprecated, use --dest 2023-10-05T21:07:40.941 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: -p [ --pool ] is deprecated, use --dest-pool 2023-10-05T21:07:40.943 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls test 2023-10-05T21:07:40.943 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep -q test3 2023-10-05T21:07:40.970 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd --image test4 -p test import /tmp/empty 2023-10-05T21:07:41.000 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:40.999+0000 7f6b0d291540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:41.010 INFO:tasks.workunit.client.0.smithi167.stderr: Importing image: 100% complete...done. 2023-10-05T21:07:41.011 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: --image is deprecated, use --dest 2023-10-05T21:07:41.011 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: -p [ --pool ] is deprecated, use --dest-pool 2023-10-05T21:07:41.013 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls test 2023-10-05T21:07:41.013 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep -q test4 2023-10-05T21:07:41.040 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd --dest test5 -p test import /tmp/empty 2023-10-05T21:07:41.071 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:41.070+0000 7ffa2c371540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:41.082 INFO:tasks.workunit.client.0.smithi167.stderr: Importing image: 100% complete...done. 2023-10-05T21:07:41.082 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: -p [ --pool ] is deprecated, use --dest-pool 2023-10-05T21:07:41.085 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls test 2023-10-05T21:07:41.085 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep -q test5 2023-10-05T21:07:41.112 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd --dest test6 --dest-pool test import /tmp/empty 2023-10-05T21:07:41.143 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:41.142+0000 7fa5fa3ff540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:41.151 INFO:tasks.workunit.client.0.smithi167.stderr: Importing image: 100% complete...done. 2023-10-05T21:07:41.154 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls test 2023-10-05T21:07:41.154 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep -q test6 2023-10-05T21:07:41.182 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd --image test7 --dest-pool test import /tmp/empty 2023-10-05T21:07:41.211 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:41.210+0000 7fd81e3f5540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:41.224 INFO:tasks.workunit.client.0.smithi167.stderr: Importing image: 100% complete...done. 2023-10-05T21:07:41.224 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: --image is deprecated, use --dest 2023-10-05T21:07:41.226 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls test 2023-10-05T21:07:41.227 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep -q test7 2023-10-05T21:07:41.254 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd --image test/test8 import /tmp/empty 2023-10-05T21:07:41.282 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:41.281+0000 7f1c35372540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:41.290 INFO:tasks.workunit.client.0.smithi167.stderr: Importing image: 100% complete...done. 2023-10-05T21:07:41.290 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: --image is deprecated, use --dest 2023-10-05T21:07:41.293 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls test 2023-10-05T21:07:41.293 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep -q test8 2023-10-05T21:07:41.320 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd --dest test/test9 import /tmp/empty 2023-10-05T21:07:41.348 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:41.347+0000 7f7a36fa9540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:41.358 INFO:tasks.workunit.client.0.smithi167.stderr: Importing image: 100% complete...done. 2023-10-05T21:07:41.361 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls test 2023-10-05T21:07:41.361 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep -q test9 2023-10-05T21:07:41.389 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd import --pool test /tmp/empty 2023-10-05T21:07:41.417 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:41.416+0000 7feb33f6f540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:41.425 INFO:tasks.workunit.client.0.smithi167.stderr: Importing image: 100% complete...done. 2023-10-05T21:07:41.425 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: -p [ --pool ] is deprecated, use --dest-pool 2023-10-05T21:07:41.428 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls test 2023-10-05T21:07:41.428 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep -q empty 2023-10-05T21:07:41.455 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd copy test/test9 test10 2023-10-05T21:07:42.620 INFO:tasks.workunit.client.0.smithi167.stderr: Image copy: 100% complete... Image copy: 100% complete...done. 2023-10-05T21:07:42.624 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls test 2023-10-05T21:07:42.624 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep -qv test10 2023-10-05T21:07:42.654 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls 2023-10-05T21:07:42.655 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep -q test10 2023-10-05T21:07:42.683 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd copy test/test9 test/test10 2023-10-05T21:07:42.733 INFO:tasks.workunit.client.0.smithi167.stderr: Image copy: 100% complete... Image copy: 100% complete...done. 2023-10-05T21:07:42.736 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls test 2023-10-05T21:07:42.736 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep -q test10 2023-10-05T21:07:42.765 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd copy --pool test test10 --dest-pool test test11 2023-10-05T21:07:42.819 INFO:tasks.workunit.client.0.smithi167.stderr: Image copy: 100% complete... Image copy: 100% complete...done. 2023-10-05T21:07:42.822 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls test 2023-10-05T21:07:42.822 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep -q test11 2023-10-05T21:07:42.850 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd copy --dest-pool rbd --pool test test11 test12 2023-10-05T21:07:42.905 INFO:tasks.workunit.client.0.smithi167.stderr: Image copy: 100% complete... Image copy: 100% complete...done. 2023-10-05T21:07:42.908 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls 2023-10-05T21:07:42.908 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep test12 2023-10-05T21:07:42.934 INFO:tasks.workunit.client.0.smithi167.stdout:test12 2023-10-05T21:07:42.935 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls test 2023-10-05T21:07:42.935 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep -qv test12 2023-10-05T21:07:42.962 INFO:tasks.workunit.client.0.smithi167.stderr:+ rm -f /tmp/empty /tmp/empty@snap 2023-10-05T21:07:42.963 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph osd pool delete test test --yes-i-really-really-mean-it 2023-10-05T21:07:43.903 INFO:tasks.workunit.client.0.smithi167.stderr:pool 'test' does not exist 2023-10-05T21:07:43.912 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in foo test1 test10 test12 test2 test3 2023-10-05T21:07:43.913 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm foo 2023-10-05T21:07:43.970 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:07:43.973 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in foo test1 test10 test12 test2 test3 2023-10-05T21:07:43.973 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test1 2023-10-05T21:07:44.011 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:07:44.014 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in foo test1 test10 test12 test2 test3 2023-10-05T21:07:44.014 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test10 2023-10-05T21:07:44.290 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:07:44.292 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in foo test1 test10 test12 test2 test3 2023-10-05T21:07:44.292 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test12 2023-10-05T21:07:44.360 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:07:44.362 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in foo test1 test10 test12 test2 test3 2023-10-05T21:07:44.362 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test2 2023-10-05T21:07:44.404 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:07:44.407 INFO:tasks.workunit.client.0.smithi167.stderr:+ for f in foo test1 test10 test12 test2 test3 2023-10-05T21:07:44.407 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test3 2023-10-05T21:07:44.446 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:07:44.449 INFO:tasks.workunit.client.0.smithi167.stderr:+ test_rename 2023-10-05T21:07:44.449 INFO:tasks.workunit.client.0.smithi167.stdout:testing rename... 2023-10-05T21:07:44.449 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'testing rename...' 2023-10-05T21:07:44.450 INFO:tasks.workunit.client.0.smithi167.stderr:+ remove_images 2023-10-05T21:07:44.450 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:44.523 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:44.602 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:44.678 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:44.752 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:44.828 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:45.098 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:45.167 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:45.236 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:45.307 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:45.376 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:45.446 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:45.522 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:45.600 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:45.678 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:45.757 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:45.833 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:45.906 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:45.975 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 1 -s 1 foo 2023-10-05T21:07:45.999 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: image format 1 is deprecated 2023-10-05T21:07:46.005 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:46.004+0000 7f5142e19540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:46.010 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 2 -s 1 bar 2023-10-05T21:07:46.051 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rename foo foo2 2023-10-05T21:07:46.093 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rename foo2 bar 2023-10-05T21:07:46.093 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep exists 2023-10-05T21:07:46.128 INFO:tasks.workunit.client.0.smithi167.stdout:2023-10-05T21:07:46.123+0000 7f70c3c32540 -1 librbd::Operations: rbd image bar already exists 2023-10-05T21:07:46.128 INFO:tasks.workunit.client.0.smithi167.stdout:rbd: rename error: (17) File exists 2023-10-05T21:07:46.128 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rename bar bar2 2023-10-05T21:07:46.188 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rename bar2 foo2 2023-10-05T21:07:46.188 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep exists 2023-10-05T21:07:46.226 INFO:tasks.workunit.client.0.smithi167.stdout:2023-10-05T21:07:46.220+0000 7fb297211540 -1 librbd::Operations: rbd image foo2 already exists 2023-10-05T21:07:46.226 INFO:tasks.workunit.client.0.smithi167.stdout:rbd: rename error: (17) File exists 2023-10-05T21:07:46.226 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph osd pool create rbd2 8 2023-10-05T21:07:46.707 INFO:tasks.workunit.client.0.smithi167.stderr:pool 'rbd2' already exists 2023-10-05T21:07:46.717 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd pool init rbd2 2023-10-05T21:07:49.594 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create -p rbd2 -s 1 foo 2023-10-05T21:07:49.629 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:49.627+0000 7f932d25c540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:51.601 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rename rbd2/foo rbd2/bar 2023-10-05T21:07:51.648 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd -p rbd2 ls 2023-10-05T21:07:51.648 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep bar 2023-10-05T21:07:51.677 INFO:tasks.workunit.client.0.smithi167.stdout:bar 2023-10-05T21:07:51.678 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rename rbd2/bar foo 2023-10-05T21:07:51.720 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rename --pool rbd2 foo bar 2023-10-05T21:07:51.767 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rename rbd2/bar --dest-pool rbd foo 2023-10-05T21:07:51.790 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: mv/rename across pools not supported 2023-10-05T21:07:51.790 INFO:tasks.workunit.client.0.smithi167.stderr:source pool: rbd2 dest pool: rbd 2023-10-05T21:07:51.791 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rename --pool rbd2 bar --dest-pool rbd2 foo 2023-10-05T21:07:51.840 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd -p rbd2 ls 2023-10-05T21:07:51.840 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep foo 2023-10-05T21:07:51.869 INFO:tasks.workunit.client.0.smithi167.stdout:foo 2023-10-05T21:07:51.869 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph osd pool rm rbd2 rbd2 --yes-i-really-really-mean-it 2023-10-05T21:07:52.719 INFO:tasks.workunit.client.0.smithi167.stderr:pool 'rbd2' does not exist 2023-10-05T21:07:52.728 INFO:tasks.workunit.client.0.smithi167.stderr:+ remove_images 2023-10-05T21:07:52.728 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:52.812 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:52.888 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:52.958 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:53.027 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:53.096 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:53.165 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:53.236 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:53.311 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:53.383 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:53.654 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:53.728 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:53.801 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:53.949 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:54.024 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:54.093 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:54.162 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:54.232 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:54.303 INFO:tasks.workunit.client.0.smithi167.stderr:+ test_ls 2023-10-05T21:07:54.304 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'testing ls...' 2023-10-05T21:07:54.304 INFO:tasks.workunit.client.0.smithi167.stdout:testing ls... 2023-10-05T21:07:54.304 INFO:tasks.workunit.client.0.smithi167.stderr:+ remove_images 2023-10-05T21:07:54.304 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:54.377 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:54.446 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:54.517 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:54.585 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:54.654 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:54.723 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:54.791 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:54.861 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:55.131 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:55.829 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:55.829 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:55.829 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:55.829 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:55.829 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:55.829 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:55.829 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:55.829 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:55.829 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 1 -s 1 test1 2023-10-05T21:07:55.829 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: image format 1 is deprecated 2023-10-05T21:07:55.830 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:55.781+0000 7facf9179540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:55.830 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 1 -s 1 test2 2023-10-05T21:07:55.830 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: image format 1 is deprecated 2023-10-05T21:07:55.830 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:55.816+0000 7fb528420540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:55.830 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls 2023-10-05T21:07:55.830 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep test1 2023-10-05T21:07:55.850 INFO:tasks.workunit.client.0.smithi167.stdout:test1 2023-10-05T21:07:55.850 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls 2023-10-05T21:07:55.850 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep test2 2023-10-05T21:07:55.877 INFO:tasks.workunit.client.0.smithi167.stdout:test2 2023-10-05T21:07:55.984 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls 2023-10-05T21:07:55.984 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:07:55.984 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 2 2023-10-05T21:07:55.984 INFO:tasks.workunit.client.0.smithi167.stdout:2 2023-10-05T21:07:55.985 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls -l 2023-10-05T21:07:55.985 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'test1.*1 MiB.*1' 2023-10-05T21:07:55.985 INFO:tasks.workunit.client.0.smithi167.stdout:test1 1 MiB 1 2023-10-05T21:07:55.986 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls -l 2023-10-05T21:07:55.986 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'test2.*1 MiB.*1' 2023-10-05T21:07:55.986 INFO:tasks.workunit.client.0.smithi167.stdout:test2 1 MiB 1 2023-10-05T21:07:55.986 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test1 2023-10-05T21:07:56.007 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:07:56.009 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test2 2023-10-05T21:07:56.045 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:07:56.048 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 2 -s 1 test1 2023-10-05T21:07:56.092 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 2 -s 1 test2 2023-10-05T21:07:56.133 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls 2023-10-05T21:07:56.133 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep test1 2023-10-05T21:07:56.160 INFO:tasks.workunit.client.0.smithi167.stdout:test1 2023-10-05T21:07:56.161 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls 2023-10-05T21:07:56.161 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep test2 2023-10-05T21:07:56.188 INFO:tasks.workunit.client.0.smithi167.stdout:test2 2023-10-05T21:07:56.188 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls 2023-10-05T21:07:56.188 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:07:56.188 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 2 2023-10-05T21:07:56.215 INFO:tasks.workunit.client.0.smithi167.stdout:2 2023-10-05T21:07:56.215 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls -l 2023-10-05T21:07:56.215 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'test1.*1 MiB.*2' 2023-10-05T21:07:56.252 INFO:tasks.workunit.client.0.smithi167.stdout:test1 1 MiB 2 2023-10-05T21:07:56.252 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls -l 2023-10-05T21:07:56.252 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'test2.*1 MiB.*2' 2023-10-05T21:07:56.288 INFO:tasks.workunit.client.0.smithi167.stdout:test2 1 MiB 2 2023-10-05T21:07:56.288 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test1 2023-10-05T21:07:56.370 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:56.368+0000 7f87e21a7700 0 -- 172.21.15.167:0/1781641563 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f87c8008740 msgr2=0x7f87c800abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:07:56.370 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:07:56.372 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test2 2023-10-05T21:07:56.454 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:56.452+0000 7f06d403f700 0 -- 172.21.15.167:0/194607290 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f06b0008740 msgr2=0x7f06b000abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:07:56.454 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:07:56.456 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 2 -s 1 test1 2023-10-05T21:07:56.500 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 1 -s 1 test2 2023-10-05T21:07:56.523 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: image format 1 is deprecated 2023-10-05T21:07:56.529 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:56.527+0000 7fb005fa2540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:56.536 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls 2023-10-05T21:07:56.537 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep test1 2023-10-05T21:07:56.563 INFO:tasks.workunit.client.0.smithi167.stdout:test1 2023-10-05T21:07:56.564 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls 2023-10-05T21:07:56.564 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep test2 2023-10-05T21:07:56.590 INFO:tasks.workunit.client.0.smithi167.stdout:test2 2023-10-05T21:07:56.591 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls 2023-10-05T21:07:56.591 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:07:56.591 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 2 2023-10-05T21:07:56.618 INFO:tasks.workunit.client.0.smithi167.stdout:2 2023-10-05T21:07:56.618 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls -l 2023-10-05T21:07:56.619 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'test1.*1 MiB.*2' 2023-10-05T21:07:56.655 INFO:tasks.workunit.client.0.smithi167.stdout:test1 1 MiB 2 2023-10-05T21:07:56.655 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls -l 2023-10-05T21:07:56.655 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'test2.*1 MiB.*1' 2023-10-05T21:07:56.690 INFO:tasks.workunit.client.0.smithi167.stdout:test2 1 MiB 1 2023-10-05T21:07:56.691 INFO:tasks.workunit.client.0.smithi167.stderr:+ remove_images 2023-10-05T21:07:56.691 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:56.760 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:56.829 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:56.898 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:56.967 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:57.037 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:57.106 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:57.174 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:57.244 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:57.515 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:57.785 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:57.854 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:57.923 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:57.993 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:58.114 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:58.186 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:58.255 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:58.325 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:07:58.395 INFO:tasks.workunit.client.0.smithi167.stderr:++ seq -w 00 99 2023-10-05T21:07:58.407 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:58.407 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.00 -s 1 2023-10-05T21:07:58.435 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:58.433+0000 7f61a267c540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:58.440 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:58.441 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.01 -s 1 2023-10-05T21:07:58.469 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:58.467+0000 7fa760e05540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:58.474 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:58.474 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.02 -s 1 2023-10-05T21:07:58.502 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:58.501+0000 7f465b7bc540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:58.507 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:58.508 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.03 -s 1 2023-10-05T21:07:58.536 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:58.534+0000 7efc82bb1540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:58.543 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:58.543 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.04 -s 1 2023-10-05T21:07:58.571 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:58.570+0000 7fcc1198a540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:58.576 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:58.577 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.05 -s 1 2023-10-05T21:07:58.607 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:58.605+0000 7f896a600540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:58.612 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:58.612 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.06 -s 1 2023-10-05T21:07:58.641 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:58.639+0000 7fb88edbe540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:58.646 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:58.646 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.07 -s 1 2023-10-05T21:07:58.674 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:58.672+0000 7f87287cb540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:58.682 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:58.682 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.08 -s 1 2023-10-05T21:07:58.710 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:58.709+0000 7fc0e895a540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:58.717 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:58.718 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.09 -s 1 2023-10-05T21:07:58.745 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:58.744+0000 7f85f40d4540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:58.751 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:58.751 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.10 -s 1 2023-10-05T21:07:58.779 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:58.778+0000 7f6fce81a540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:58.785 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:58.785 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.11 -s 1 2023-10-05T21:07:58.814 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:58.812+0000 7f0d6646e540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:58.819 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:58.819 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.12 -s 1 2023-10-05T21:07:58.846 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:58.845+0000 7fed47a7a540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:58.852 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:58.852 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.13 -s 1 2023-10-05T21:07:58.880 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:58.879+0000 7f69add02540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:58.886 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:58.886 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.14 -s 1 2023-10-05T21:07:58.913 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:58.911+0000 7f91d1c1c540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:58.919 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:58.919 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.15 -s 1 2023-10-05T21:07:58.947 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:58.945+0000 7f196ad90540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:58.953 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:58.953 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.16 -s 1 2023-10-05T21:07:58.980 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:58.979+0000 7f95a145d540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:58.989 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:58.989 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.17 -s 1 2023-10-05T21:07:59.017 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:59.015+0000 7f480d6ac540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:59.024 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:59.024 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.18 -s 1 2023-10-05T21:07:59.052 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:59.050+0000 7feccafe2540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:59.060 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:59.061 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.19 -s 1 2023-10-05T21:07:59.089 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:59.087+0000 7f06dfa90540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:59.094 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:59.095 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.20 -s 1 2023-10-05T21:07:59.122 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:59.121+0000 7fdb4941c540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:59.127 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:59.127 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.21 -s 1 2023-10-05T21:07:59.156 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:59.154+0000 7f1a37c1c540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:59.161 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:59.161 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.22 -s 1 2023-10-05T21:07:59.189 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:59.187+0000 7ff3708c4540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:59.197 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:59.197 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.23 -s 1 2023-10-05T21:07:59.225 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:59.223+0000 7f478324b540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:59.232 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:59.232 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.24 -s 1 2023-10-05T21:07:59.260 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:59.259+0000 7f00e1082540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:59.267 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:59.267 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.25 -s 1 2023-10-05T21:07:59.295 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:59.293+0000 7f01e2471540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:59.301 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:59.301 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.26 -s 1 2023-10-05T21:07:59.329 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:59.327+0000 7fd578d01540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:59.336 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:59.336 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.27 -s 1 2023-10-05T21:07:59.365 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:59.363+0000 7f0e1724d540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:59.372 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:59.372 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.28 -s 1 2023-10-05T21:07:59.400 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:59.398+0000 7ff542ec4540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:59.407 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:59.407 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.29 -s 1 2023-10-05T21:07:59.435 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:59.433+0000 7ff887f85540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:59.440 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:59.440 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.30 -s 1 2023-10-05T21:07:59.469 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:59.467+0000 7f175af9f540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:59.475 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:59.475 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.31 -s 1 2023-10-05T21:07:59.502 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:59.501+0000 7fe13493c540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:59.508 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:59.508 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.32 -s 1 2023-10-05T21:07:59.537 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:59.535+0000 7fef7d126540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:59.542 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:59.542 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.33 -s 1 2023-10-05T21:07:59.570 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:59.568+0000 7f3a8fdca540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:59.575 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:59.575 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.34 -s 1 2023-10-05T21:07:59.603 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:59.602+0000 7fc72771b540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:59.609 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:59.609 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.35 -s 1 2023-10-05T21:07:59.638 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:59.636+0000 7f191d097540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:59.643 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:59.643 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.36 -s 1 2023-10-05T21:07:59.671 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:59.669+0000 7f8b01481540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:59.676 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:59.676 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.37 -s 1 2023-10-05T21:07:59.704 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:59.703+0000 7f0df3510540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:59.709 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:59.710 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.38 -s 1 2023-10-05T21:07:59.738 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:59.736+0000 7f221fb48540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:59.744 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:59.744 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.39 -s 1 2023-10-05T21:07:59.772 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:59.771+0000 7fa42c7e5540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:59.777 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:59.778 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.40 -s 1 2023-10-05T21:07:59.806 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:59.804+0000 7fd8ab885540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:59.811 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:59.812 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.41 -s 1 2023-10-05T21:07:59.840 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:59.838+0000 7f4acf7c5540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:59.846 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:59.846 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.42 -s 1 2023-10-05T21:07:59.874 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:59.872+0000 7f16c5b1c540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:59.880 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:59.880 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.43 -s 1 2023-10-05T21:07:59.908 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:59.906+0000 7f16e4919540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:59.913 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:59.913 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.44 -s 1 2023-10-05T21:07:59.942 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:59.940+0000 7f0d640d7540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:59.947 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:59.947 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.45 -s 1 2023-10-05T21:07:59.975 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:07:59.973+0000 7fb4c5c97540 -1 librbd: Forced V1 image creation. 2023-10-05T21:07:59.981 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:07:59.981 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.46 -s 1 2023-10-05T21:08:00.009 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:00.007+0000 7f1989b87540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:00.015 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:00.015 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.47 -s 1 2023-10-05T21:08:00.043 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:00.042+0000 7f3615fea540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:00.051 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:00.051 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.48 -s 1 2023-10-05T21:08:00.079 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:00.077+0000 7fedd9026540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:00.084 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:00.085 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.49 -s 1 2023-10-05T21:08:00.112 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:00.111+0000 7f4ff84eb540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:00.118 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:00.118 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.50 -s 1 2023-10-05T21:08:00.146 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:00.144+0000 7f13968ef540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:00.151 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:00.152 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.51 -s 1 2023-10-05T21:08:00.180 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:00.178+0000 7f7cf4e9b540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:00.187 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:00.187 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.52 -s 1 2023-10-05T21:08:00.215 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:00.213+0000 7f67f2a0d540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:00.221 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:00.221 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.53 -s 1 2023-10-05T21:08:00.249 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:00.247+0000 7fe6546b1540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:00.255 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:00.255 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.54 -s 1 2023-10-05T21:08:00.284 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:00.282+0000 7ff8fc05c540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:00.290 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:00.290 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.55 -s 1 2023-10-05T21:08:00.319 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:00.317+0000 7f877d62b540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:00.326 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:00.326 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.56 -s 1 2023-10-05T21:08:00.355 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:00.353+0000 7f9be2739540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:00.361 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:00.361 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.57 -s 1 2023-10-05T21:08:00.390 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:00.389+0000 7f4765419540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:00.396 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:00.396 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.58 -s 1 2023-10-05T21:08:00.424 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:00.423+0000 7fd18760b540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:00.432 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:00.432 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.59 -s 1 2023-10-05T21:08:00.460 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:00.458+0000 7fc98278b540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:00.466 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:00.466 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.60 -s 1 2023-10-05T21:08:00.495 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:00.493+0000 7fb3d504d540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:00.502 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:00.502 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.61 -s 1 2023-10-05T21:08:00.531 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:00.529+0000 7ffa09f47540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:00.541 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:00.541 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.62 -s 1 2023-10-05T21:08:00.567 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:00.565+0000 7fae6a712540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:00.572 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:00.572 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.63 -s 1 2023-10-05T21:08:00.601 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:00.599+0000 7ff9d9519540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:00.609 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:00.610 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.64 -s 1 2023-10-05T21:08:00.639 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:00.637+0000 7f2e4cb34540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:00.648 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:00.648 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.65 -s 1 2023-10-05T21:08:00.676 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:00.674+0000 7fa302d50540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:00.683 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:00.683 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.66 -s 1 2023-10-05T21:08:00.711 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:00.709+0000 7fd9c8fbf540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:00.718 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:00.718 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.67 -s 1 2023-10-05T21:08:00.747 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:00.745+0000 7fee091e6540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:00.755 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:00.755 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.68 -s 1 2023-10-05T21:08:00.784 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:00.782+0000 7fac65338540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:00.792 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:00.792 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.69 -s 1 2023-10-05T21:08:00.819 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:00.818+0000 7f5e2917d540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:00.825 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:00.825 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.70 -s 1 2023-10-05T21:08:00.853 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:00.852+0000 7f6365aef540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:00.865 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:00.865 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.71 -s 1 2023-10-05T21:08:00.893 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:00.892+0000 7ff646182540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:00.899 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:00.900 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.72 -s 1 2023-10-05T21:08:00.928 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:00.926+0000 7f329169a540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:00.933 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:00.933 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.73 -s 1 2023-10-05T21:08:00.962 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:00.960+0000 7f79feb01540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:00.969 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:00.970 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.74 -s 1 2023-10-05T21:08:00.998 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:00.996+0000 7fee98af1540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:01.004 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:01.004 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.75 -s 1 2023-10-05T21:08:01.032 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:01.030+0000 7ffb722f4540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:01.038 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:01.038 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.76 -s 1 2023-10-05T21:08:01.066 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:01.064+0000 7f6f51d9b540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:01.072 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:01.072 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.77 -s 1 2023-10-05T21:08:01.100 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:01.098+0000 7f6664fd7540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:01.109 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:01.109 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.78 -s 1 2023-10-05T21:08:01.137 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:01.135+0000 7fe187e75540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:01.142 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:01.142 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.79 -s 1 2023-10-05T21:08:01.171 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:01.169+0000 7f1161b57540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:01.181 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:01.182 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.80 -s 1 2023-10-05T21:08:01.210 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:01.209+0000 7f17475e1540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:01.216 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:01.216 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.81 -s 1 2023-10-05T21:08:01.245 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:01.243+0000 7f18e3d16540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:01.251 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:01.251 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.82 -s 1 2023-10-05T21:08:01.280 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:01.278+0000 7f7031f0f540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:01.287 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:01.287 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.83 -s 1 2023-10-05T21:08:01.315 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:01.313+0000 7fb15c632540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:01.321 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:01.321 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.84 -s 1 2023-10-05T21:08:01.349 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:01.347+0000 7fd747ed9540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:01.355 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:01.355 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.85 -s 1 2023-10-05T21:08:01.383 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:01.381+0000 7feab8b68540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:01.390 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:01.390 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.86 -s 1 2023-10-05T21:08:01.419 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:01.417+0000 7f79bba11540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:01.426 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:01.426 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.87 -s 1 2023-10-05T21:08:01.455 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:01.453+0000 7f5bb4713540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:01.467 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:01.467 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.88 -s 1 2023-10-05T21:08:01.495 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:01.493+0000 7fa15daff540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:01.501 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:01.501 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.89 -s 1 2023-10-05T21:08:01.529 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:01.527+0000 7f6afb9d3540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:01.538 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:01.538 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.90 -s 1 2023-10-05T21:08:01.566 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:01.564+0000 7f49d529f540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:01.575 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:01.575 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.91 -s 1 2023-10-05T21:08:01.605 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:01.603+0000 7fd46a62b540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:01.611 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:01.611 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.92 -s 1 2023-10-05T21:08:01.639 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:01.637+0000 7ff21af7c540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:01.645 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:01.645 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.93 -s 1 2023-10-05T21:08:01.674 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:01.672+0000 7feb1c7fa540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:01.680 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:01.680 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.94 -s 1 2023-10-05T21:08:01.708 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:01.707+0000 7fd3968b9540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:01.714 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:01.714 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.95 -s 1 2023-10-05T21:08:01.943 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:01.941+0000 7f3e48707540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:01.949 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:01.949 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.96 -s 1 2023-10-05T21:08:01.978 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:01.976+0000 7f443e0b9540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:01.983 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:01.983 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.97 -s 1 2023-10-05T21:08:02.012 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:02.010+0000 7f23dd0db540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:02.017 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:02.017 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.98 -s 1 2023-10-05T21:08:02.045 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:02.044+0000 7f0726f9a540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:02.053 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:02.053 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.99 -s 1 2023-10-05T21:08:02.081 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:02.079+0000 7f3c86355540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:02.090 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls 2023-10-05T21:08:02.090 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:08:02.090 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 100 2023-10-05T21:08:02.116 INFO:tasks.workunit.client.0.smithi167.stdout:100 2023-10-05T21:08:02.117 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls -l 2023-10-05T21:08:02.117 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep image 2023-10-05T21:08:02.117 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:08:02.117 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 100 2023-10-05T21:08:02.182 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:02.180+0000 7f3aeddaa700 0 -- 172.21.15.167:0/1599161229 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x559461da6820 msgr2=0x559461dacc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:02.194 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:02.192+0000 7f3aeddaa700 0 -- 172.21.15.167:0/1599161229 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f3ad003a330 msgr2=0x7f3ad003c710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:02.373 INFO:tasks.workunit.client.0.smithi167.stdout:100 2023-10-05T21:08:02.373 INFO:tasks.workunit.client.0.smithi167.stderr:++ seq -w 00 99 2023-10-05T21:08:02.374 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:02.374 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.00 2023-10-05T21:08:02.414 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:02.416 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:02.416 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.01 2023-10-05T21:08:02.452 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:02.455 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:02.455 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.02 2023-10-05T21:08:02.492 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:02.494 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:02.494 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.03 2023-10-05T21:08:02.532 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:02.534 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:02.535 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.04 2023-10-05T21:08:02.570 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:02.573 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:02.573 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.05 2023-10-05T21:08:02.612 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:02.615 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:02.615 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.06 2023-10-05T21:08:02.652 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:02.655 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:02.655 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.07 2023-10-05T21:08:02.691 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:02.693 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:02.694 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.08 2023-10-05T21:08:02.730 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:02.732 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:02.733 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.09 2023-10-05T21:08:02.768 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:02.770 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:02.770 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.10 2023-10-05T21:08:02.815 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:02.817 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:02.818 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.11 2023-10-05T21:08:02.853 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:02.855 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:02.856 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.12 2023-10-05T21:08:02.891 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:02.894 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:02.894 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.13 2023-10-05T21:08:02.930 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:02.932 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:02.933 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.14 2023-10-05T21:08:02.969 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:02.972 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:02.972 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.15 2023-10-05T21:08:03.007 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:03.010 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:03.010 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.16 2023-10-05T21:08:03.045 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:03.048 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:03.048 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.17 2023-10-05T21:08:03.084 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:03.086 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:03.086 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.18 2023-10-05T21:08:03.122 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:03.125 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:03.125 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.19 2023-10-05T21:08:03.160 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:03.163 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:03.163 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.20 2023-10-05T21:08:03.198 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:03.201 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:03.201 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.21 2023-10-05T21:08:03.236 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:03.238 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:03.238 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.22 2023-10-05T21:08:03.274 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:03.276 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:03.276 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.23 2023-10-05T21:08:03.314 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:03.317 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:03.317 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.24 2023-10-05T21:08:03.352 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:03.355 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:03.355 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.25 2023-10-05T21:08:03.391 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:03.394 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:03.394 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.26 2023-10-05T21:08:03.430 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:03.432 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:03.432 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.27 2023-10-05T21:08:03.468 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:03.470 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:03.470 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.28 2023-10-05T21:08:03.506 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:03.508 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:03.508 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.29 2023-10-05T21:08:03.545 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:03.548 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:03.548 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.30 2023-10-05T21:08:03.584 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:03.587 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:03.587 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.31 2023-10-05T21:08:03.622 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:03.625 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:03.625 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.32 2023-10-05T21:08:03.666 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:03.669 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:03.669 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.33 2023-10-05T21:08:03.704 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:03.707 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:03.707 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.34 2023-10-05T21:08:03.748 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:03.751 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:03.751 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.35 2023-10-05T21:08:03.787 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:03.789 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:03.789 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.36 2023-10-05T21:08:03.826 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:03.828 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:03.828 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.37 2023-10-05T21:08:03.867 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:03.870 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:03.870 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.38 2023-10-05T21:08:03.906 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:03.908 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:03.908 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.39 2023-10-05T21:08:03.944 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:03.946 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:03.946 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.40 2023-10-05T21:08:03.982 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:03.985 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:03.985 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.41 2023-10-05T21:08:04.026 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:04.028 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:04.028 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.42 2023-10-05T21:08:04.069 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:04.072 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:04.072 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.43 2023-10-05T21:08:04.112 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:04.115 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:04.115 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.44 2023-10-05T21:08:04.150 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:04.152 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:04.152 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.45 2023-10-05T21:08:04.188 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:04.191 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:04.191 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.46 2023-10-05T21:08:04.227 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:04.230 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:04.230 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.47 2023-10-05T21:08:04.267 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:04.269 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:04.269 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.48 2023-10-05T21:08:04.311 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:04.314 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:04.314 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.49 2023-10-05T21:08:04.350 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:04.353 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:04.353 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.50 2023-10-05T21:08:04.388 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:04.390 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:04.391 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.51 2023-10-05T21:08:04.427 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:04.429 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:04.429 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.52 2023-10-05T21:08:04.465 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:04.467 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:04.468 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.53 2023-10-05T21:08:04.508 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:04.510 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:04.510 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.54 2023-10-05T21:08:04.546 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:04.548 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:04.548 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.55 2023-10-05T21:08:04.585 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:04.588 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:04.588 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.56 2023-10-05T21:08:04.623 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:04.626 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:04.626 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.57 2023-10-05T21:08:04.662 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:04.664 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:04.665 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.58 2023-10-05T21:08:04.700 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:04.702 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:04.702 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.59 2023-10-05T21:08:04.738 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:04.741 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:04.741 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.60 2023-10-05T21:08:04.777 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:04.780 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:04.780 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.61 2023-10-05T21:08:04.819 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:04.822 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:04.822 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.62 2023-10-05T21:08:04.857 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:04.860 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:04.860 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.63 2023-10-05T21:08:04.901 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:04.904 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:04.904 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.64 2023-10-05T21:08:04.940 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:04.943 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:04.943 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.65 2023-10-05T21:08:04.979 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:04.981 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:04.982 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.66 2023-10-05T21:08:05.019 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:05.021 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:05.021 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.67 2023-10-05T21:08:05.059 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:05.062 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:05.062 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.68 2023-10-05T21:08:05.101 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:05.104 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:05.104 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.69 2023-10-05T21:08:05.140 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:05.142 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:05.142 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.70 2023-10-05T21:08:05.178 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:05.181 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:05.181 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.71 2023-10-05T21:08:05.221 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:05.223 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:05.223 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.72 2023-10-05T21:08:05.259 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:05.261 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:05.261 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.73 2023-10-05T21:08:05.297 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:05.299 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:05.300 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.74 2023-10-05T21:08:05.335 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:05.337 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:05.337 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.75 2023-10-05T21:08:05.373 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:05.375 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:05.375 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.76 2023-10-05T21:08:05.417 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:05.419 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:05.419 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.77 2023-10-05T21:08:05.455 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:05.458 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:05.458 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.78 2023-10-05T21:08:05.494 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:05.497 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:05.497 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.79 2023-10-05T21:08:05.533 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:05.535 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:05.535 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.80 2023-10-05T21:08:05.571 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:05.573 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:05.573 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.81 2023-10-05T21:08:05.609 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:05.611 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:05.611 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.82 2023-10-05T21:08:05.650 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:05.653 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:05.653 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.83 2023-10-05T21:08:05.690 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:05.692 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:05.692 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.84 2023-10-05T21:08:05.728 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:05.731 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:05.731 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.85 2023-10-05T21:08:05.766 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:05.769 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:05.769 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.86 2023-10-05T21:08:05.804 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:05.807 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:05.807 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.87 2023-10-05T21:08:05.843 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:05.846 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:05.846 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.88 2023-10-05T21:08:05.882 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:05.884 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:05.885 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.89 2023-10-05T21:08:05.922 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:05.925 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:05.925 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.90 2023-10-05T21:08:05.968 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:05.970 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:05.970 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.91 2023-10-05T21:08:06.006 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:06.008 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:06.008 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.92 2023-10-05T21:08:06.047 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:06.050 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:06.050 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.93 2023-10-05T21:08:06.089 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:06.091 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:06.091 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.94 2023-10-05T21:08:06.128 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:06.130 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:06.130 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.95 2023-10-05T21:08:06.168 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:06.171 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:06.171 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.96 2023-10-05T21:08:06.207 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:06.210 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:06.210 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.97 2023-10-05T21:08:06.245 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:06.248 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:06.248 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.98 2023-10-05T21:08:06.287 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:06.289 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:06.289 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.99 2023-10-05T21:08:06.329 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:06.332 INFO:tasks.workunit.client.0.smithi167.stderr:++ seq -w 00 99 2023-10-05T21:08:06.333 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:06.333 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.00 --image-format 2 -s 1 2023-10-05T21:08:06.381 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:06.381 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.01 --image-format 2 -s 1 2023-10-05T21:08:06.421 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:06.421 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.02 --image-format 2 -s 1 2023-10-05T21:08:06.463 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:06.463 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.03 --image-format 2 -s 1 2023-10-05T21:08:06.705 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:06.705 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.04 --image-format 2 -s 1 2023-10-05T21:08:06.747 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:06.748 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.05 --image-format 2 -s 1 2023-10-05T21:08:06.787 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:06.788 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.06 --image-format 2 -s 1 2023-10-05T21:08:06.831 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:06.831 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.07 --image-format 2 -s 1 2023-10-05T21:08:06.873 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:06.873 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.08 --image-format 2 -s 1 2023-10-05T21:08:06.914 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:06.914 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.09 --image-format 2 -s 1 2023-10-05T21:08:06.957 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:06.957 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.10 --image-format 2 -s 1 2023-10-05T21:08:07.000 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:07.000 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.11 --image-format 2 -s 1 2023-10-05T21:08:07.040 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:07.040 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.12 --image-format 2 -s 1 2023-10-05T21:08:07.084 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:07.084 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.13 --image-format 2 -s 1 2023-10-05T21:08:07.125 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:07.125 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.14 --image-format 2 -s 1 2023-10-05T21:08:07.166 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:07.167 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.15 --image-format 2 -s 1 2023-10-05T21:08:07.208 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:07.208 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.16 --image-format 2 -s 1 2023-10-05T21:08:07.248 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:07.249 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.17 --image-format 2 -s 1 2023-10-05T21:08:07.289 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:07.290 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.18 --image-format 2 -s 1 2023-10-05T21:08:07.330 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:07.331 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.19 --image-format 2 -s 1 2023-10-05T21:08:07.372 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:07.373 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.20 --image-format 2 -s 1 2023-10-05T21:08:07.414 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:07.414 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.21 --image-format 2 -s 1 2023-10-05T21:08:07.455 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:07.456 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.22 --image-format 2 -s 1 2023-10-05T21:08:07.497 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:07.497 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.23 --image-format 2 -s 1 2023-10-05T21:08:07.537 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:07.537 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.24 --image-format 2 -s 1 2023-10-05T21:08:07.579 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:07.579 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.25 --image-format 2 -s 1 2023-10-05T21:08:07.624 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:07.624 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.26 --image-format 2 -s 1 2023-10-05T21:08:07.664 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:07.664 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.27 --image-format 2 -s 1 2023-10-05T21:08:07.705 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:07.706 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.28 --image-format 2 -s 1 2023-10-05T21:08:07.749 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:07.749 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.29 --image-format 2 -s 1 2023-10-05T21:08:07.789 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:07.789 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.30 --image-format 2 -s 1 2023-10-05T21:08:07.830 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:07.830 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.31 --image-format 2 -s 1 2023-10-05T21:08:07.872 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:07.872 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.32 --image-format 2 -s 1 2023-10-05T21:08:07.912 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:07.912 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.33 --image-format 2 -s 1 2023-10-05T21:08:07.955 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:07.955 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.34 --image-format 2 -s 1 2023-10-05T21:08:07.995 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:07.996 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.35 --image-format 2 -s 1 2023-10-05T21:08:08.036 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:08.037 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.36 --image-format 2 -s 1 2023-10-05T21:08:08.077 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:08.078 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.37 --image-format 2 -s 1 2023-10-05T21:08:08.120 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:08.121 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.38 --image-format 2 -s 1 2023-10-05T21:08:08.162 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:08.162 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.39 --image-format 2 -s 1 2023-10-05T21:08:08.205 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:08.205 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.40 --image-format 2 -s 1 2023-10-05T21:08:08.246 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:08.246 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.41 --image-format 2 -s 1 2023-10-05T21:08:08.288 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:08.288 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.42 --image-format 2 -s 1 2023-10-05T21:08:08.331 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:08.331 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.43 --image-format 2 -s 1 2023-10-05T21:08:08.371 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:08.371 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.44 --image-format 2 -s 1 2023-10-05T21:08:08.412 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:08.412 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.45 --image-format 2 -s 1 2023-10-05T21:08:08.455 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:08.455 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.46 --image-format 2 -s 1 2023-10-05T21:08:08.498 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:08.498 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.47 --image-format 2 -s 1 2023-10-05T21:08:08.540 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:08.540 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.48 --image-format 2 -s 1 2023-10-05T21:08:08.581 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:08.581 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.49 --image-format 2 -s 1 2023-10-05T21:08:08.624 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:08.625 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.50 --image-format 2 -s 1 2023-10-05T21:08:08.667 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:08.667 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.51 --image-format 2 -s 1 2023-10-05T21:08:08.708 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:08.708 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.52 --image-format 2 -s 1 2023-10-05T21:08:08.750 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:08.750 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.53 --image-format 2 -s 1 2023-10-05T21:08:08.791 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:08.792 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.54 --image-format 2 -s 1 2023-10-05T21:08:08.833 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:08.833 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.55 --image-format 2 -s 1 2023-10-05T21:08:08.876 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:08.876 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.56 --image-format 2 -s 1 2023-10-05T21:08:08.917 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:08.917 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.57 --image-format 2 -s 1 2023-10-05T21:08:08.959 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:08.959 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.58 --image-format 2 -s 1 2023-10-05T21:08:09.002 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:09.002 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.59 --image-format 2 -s 1 2023-10-05T21:08:09.044 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:09.044 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.60 --image-format 2 -s 1 2023-10-05T21:08:09.087 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:09.087 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.61 --image-format 2 -s 1 2023-10-05T21:08:09.128 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:09.128 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.62 --image-format 2 -s 1 2023-10-05T21:08:09.168 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:09.169 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.63 --image-format 2 -s 1 2023-10-05T21:08:09.210 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:09.210 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.64 --image-format 2 -s 1 2023-10-05T21:08:09.252 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:09.252 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.65 --image-format 2 -s 1 2023-10-05T21:08:09.294 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:09.294 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.66 --image-format 2 -s 1 2023-10-05T21:08:09.335 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:09.336 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.67 --image-format 2 -s 1 2023-10-05T21:08:09.376 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:09.376 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.68 --image-format 2 -s 1 2023-10-05T21:08:09.420 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:09.420 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.69 --image-format 2 -s 1 2023-10-05T21:08:09.461 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:09.461 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.70 --image-format 2 -s 1 2023-10-05T21:08:09.504 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:09.504 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.71 --image-format 2 -s 1 2023-10-05T21:08:09.547 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:09.547 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.72 --image-format 2 -s 1 2023-10-05T21:08:09.591 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:09.592 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.73 --image-format 2 -s 1 2023-10-05T21:08:09.634 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:09.634 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.74 --image-format 2 -s 1 2023-10-05T21:08:09.677 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:09.677 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.75 --image-format 2 -s 1 2023-10-05T21:08:09.719 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:09.719 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.76 --image-format 2 -s 1 2023-10-05T21:08:09.762 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:09.762 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.77 --image-format 2 -s 1 2023-10-05T21:08:09.803 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:09.803 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.78 --image-format 2 -s 1 2023-10-05T21:08:09.844 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:09.844 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.79 --image-format 2 -s 1 2023-10-05T21:08:09.884 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:09.885 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.80 --image-format 2 -s 1 2023-10-05T21:08:09.925 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:09.925 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.81 --image-format 2 -s 1 2023-10-05T21:08:09.967 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:09.967 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.82 --image-format 2 -s 1 2023-10-05T21:08:10.008 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:10.009 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.83 --image-format 2 -s 1 2023-10-05T21:08:10.049 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:10.049 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.84 --image-format 2 -s 1 2023-10-05T21:08:10.094 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:10.094 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.85 --image-format 2 -s 1 2023-10-05T21:08:10.135 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:10.135 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.86 --image-format 2 -s 1 2023-10-05T21:08:10.178 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:10.179 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.87 --image-format 2 -s 1 2023-10-05T21:08:10.218 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:10.218 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.88 --image-format 2 -s 1 2023-10-05T21:08:10.259 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:10.259 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.89 --image-format 2 -s 1 2023-10-05T21:08:10.300 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:10.300 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.90 --image-format 2 -s 1 2023-10-05T21:08:10.342 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:10.342 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.91 --image-format 2 -s 1 2023-10-05T21:08:10.384 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:10.385 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.92 --image-format 2 -s 1 2023-10-05T21:08:10.425 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:10.426 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.93 --image-format 2 -s 1 2023-10-05T21:08:10.466 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:10.467 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.94 --image-format 2 -s 1 2023-10-05T21:08:10.509 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:10.509 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.95 --image-format 2 -s 1 2023-10-05T21:08:10.551 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:10.551 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.96 --image-format 2 -s 1 2023-10-05T21:08:10.591 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:10.592 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.97 --image-format 2 -s 1 2023-10-05T21:08:10.633 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:10.633 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.98 --image-format 2 -s 1 2023-10-05T21:08:10.675 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:10.675 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create image.99 --image-format 2 -s 1 2023-10-05T21:08:10.719 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls 2023-10-05T21:08:10.719 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:08:10.719 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 100 2023-10-05T21:08:10.748 INFO:tasks.workunit.client.0.smithi167.stdout:100 2023-10-05T21:08:10.748 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls -l 2023-10-05T21:08:10.748 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep image 2023-10-05T21:08:10.749 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:08:10.749 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 100 2023-10-05T21:08:10.806 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:10.804+0000 7fb064a29700 0 -- 172.21.15.167:0/1065322284 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x56179793c320 msgr2=0x56179794cce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:10.815 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:10.813+0000 7fb064a29700 0 -- 172.21.15.167:0/1065322284 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x7fb04803a330 msgr2=0x7fb04803c710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:11.089 INFO:tasks.workunit.client.0.smithi167.stdout:100 2023-10-05T21:08:11.089 INFO:tasks.workunit.client.0.smithi167.stderr:++ seq -w 00 99 2023-10-05T21:08:11.090 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:11.090 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.00 2023-10-05T21:08:11.177 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:11.175+0000 7f6afc291700 0 -- 172.21.15.167:0/2402017660 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f6adc03a250 msgr2=0x7f6adc03c630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:11.183 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:11.186 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:11.186 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.01 2023-10-05T21:08:11.271 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:11.269+0000 7ff2c8585700 0 -- 172.21.15.167:0/748472571 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x55d956dfaf10 msgr2=0x55d956dd35c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:11.271 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:11.274 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:11.274 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.02 2023-10-05T21:08:11.361 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:11.360+0000 7f0d17c97700 0 -- 172.21.15.167:0/3265595685 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f0cf4008740 msgr2=0x7f0cf400abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:11.362 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:11.365 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:11.365 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.03 2023-10-05T21:08:11.454 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:11.452+0000 7f0a78c09700 0 -- 172.21.15.167:0/2064758512 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f0a54008740 msgr2=0x7f0a5400abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-05T21:08:11.455 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:11.458 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:11.458 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.04 2023-10-05T21:08:11.547 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:11.544+0000 7f05b632e700 0 -- 172.21.15.167:0/606005167 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f059c008740 msgr2=0x7f059c00abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:11.547 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:11.549 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:11.550 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.05 2023-10-05T21:08:11.633 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:11.631+0000 7f2aea92e700 0 -- 172.21.15.167:0/2327419550 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x555f479afc40 msgr2=0x555f479ac880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:11.634 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:11.636 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:11.637 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.06 2023-10-05T21:08:11.720 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:11.718+0000 7fc7dd1c6700 0 -- 172.21.15.167:0/4104157368 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7fc7b8008740 msgr2=0x7fc7b800abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:11.721 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:11.723 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:11.724 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.07 2023-10-05T21:08:11.818 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:11.816+0000 7f4fdc2d8700 0 -- 172.21.15.167:0/3697981802 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f4fb8006500 msgr2=0x7f4fb8011eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:11.818 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:11.820 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:11.820 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.08 2023-10-05T21:08:11.906 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:11.904+0000 7f11e5a57700 0 -- 172.21.15.167:0/655076405 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f11cc008740 msgr2=0x7f11cc00abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:11.906 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:11.908 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:11.909 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.09 2023-10-05T21:08:11.992 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:11.990+0000 7f8e17f4e700 0 -- 172.21.15.167:0/1228418169 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f8df4008740 msgr2=0x7f8df400abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-05T21:08:11.992 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:11.995 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:11.995 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.10 2023-10-05T21:08:12.082 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:12.080+0000 7fd6755ee700 0 -- 172.21.15.167:0/3888303766 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x557c82f89480 msgr2=0x557c82f75de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-05T21:08:12.082 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:12.085 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:12.085 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.11 2023-10-05T21:08:12.170 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:12.168+0000 7f311978e700 0 -- 172.21.15.167:0/1790975083 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x563292fa30b0 msgr2=0x563292f77b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:12.170 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:12.173 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:12.173 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.12 2023-10-05T21:08:12.256 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:12.254+0000 7f8f15a5b700 0 -- 172.21.15.167:0/3056897276 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f8efc008740 msgr2=0x7f8efc00abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-05T21:08:12.257 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:12.260 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:12.260 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.13 2023-10-05T21:08:12.341 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:12.339+0000 7f24bb1ff700 0 -- 172.21.15.167:0/2272618793 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f24a0008740 msgr2=0x7f24a000abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:12.341 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:12.344 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:12.344 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.14 2023-10-05T21:08:12.434 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:12.433+0000 7ff519e84700 0 -- 172.21.15.167:0/4188855007 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7ff4f8008740 msgr2=0x7ff4f800abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-05T21:08:12.436 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:12.439 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:12.439 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.15 2023-10-05T21:08:12.525 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:12.524+0000 7f004aea6700 0 -- 172.21.15.167:0/2212131420 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x55f420619c90 msgr2=0x55f42063b200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:12.526 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:12.529 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:12.529 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.16 2023-10-05T21:08:12.615 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:12.613+0000 7fb0fbed2700 0 -- 172.21.15.167:0/1771718874 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7fb0dc006720 msgr2=0x7fb0dc008ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:12.617 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:12.617 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:12.618 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.17 2023-10-05T21:08:12.704 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:12.702+0000 7fd07dfcc700 0 -- 172.21.15.167:0/877108561 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7fd064008740 msgr2=0x7fd06400abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:12.704 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:12.706 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:12.706 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.18 2023-10-05T21:08:12.790 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:12.788+0000 7f5f1e432700 0 -- 172.21.15.167:0/1633741519 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f5f04006500 msgr2=0x7f5f04011eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:12.790 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:12.793 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:12.793 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.19 2023-10-05T21:08:12.877 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:12.874+0000 7f95c9a95700 0 -- 172.21.15.167:0/121278760 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f95a8008740 msgr2=0x7f95a800abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:12.877 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:12.879 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:12.879 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.20 2023-10-05T21:08:12.961 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:12.959+0000 7f6a89ab1700 0 -- 172.21.15.167:0/1222875685 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x560ac4dd7d40 msgr2=0x560ac4d8d6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:12.962 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:12.965 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:12.965 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.21 2023-10-05T21:08:13.044 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:13.042+0000 7fc82cbe6700 0 -- 172.21.15.167:0/2477652806 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x7fc80c03a330 msgr2=0x7fc80c03c710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:13.055 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:13.058 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:13.058 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.22 2023-10-05T21:08:13.149 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:13.146+0000 7fca3e925700 0 -- 172.21.15.167:0/3026867980 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7fca1c008740 msgr2=0x7fca1c00abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:13.149 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:13.152 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:13.152 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.23 2023-10-05T21:08:13.233 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:13.231+0000 7f183d876700 0 -- 172.21.15.167:0/1985701100 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x55722d0de560 msgr2=0x55722d12ee60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:13.233 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:13.235 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:13.238 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.24 2023-10-05T21:08:13.319 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:13.317+0000 7f60c23a8700 0 -- 172.21.15.167:0/2789459825 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f609c006720 msgr2=0x7f609c008ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:13.319 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:13.322 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:13.322 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.25 2023-10-05T21:08:13.410 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:13.408+0000 7fc030733700 0 -- 172.21.15.167:0/3274515904 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7fc014008740 msgr2=0x7fc01400abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:13.411 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:13.413 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:13.414 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.26 2023-10-05T21:08:13.500 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:13.498+0000 7f19b9b47700 0 -- 172.21.15.167:0/2144890707 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f1994005fd0 msgr2=0x7f1994008460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:13.500 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:13.503 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:13.503 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.27 2023-10-05T21:08:13.593 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:13.591+0000 7f12bc753700 0 -- 172.21.15.167:0/532022100 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f12a003a2e0 msgr2=0x7f12a003c6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:13.598 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:13.600 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:13.600 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.28 2023-10-05T21:08:13.685 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:13.683+0000 7f50f02a2700 0 -- 172.21.15.167:0/2840035702 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f50cc008740 msgr2=0x7f50cc00abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:13.686 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:13.689 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:13.689 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.29 2023-10-05T21:08:13.778 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:13.776+0000 7feb93eb1700 0 -- 172.21.15.167:0/3558426854 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7feb70008740 msgr2=0x7feb7000abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:13.779 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:13.781 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:13.781 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.30 2023-10-05T21:08:13.868 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:13.866+0000 7f549fb35700 0 -- 172.21.15.167:0/2056894094 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f5484006500 msgr2=0x7f5484011eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:13.868 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:13.871 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:13.871 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.31 2023-10-05T21:08:13.956 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:13.953+0000 7f74d9195700 0 -- 172.21.15.167:0/3683762643 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f74b4006510 msgr2=0x7f74b4011ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:13.956 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:13.958 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:13.958 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.32 2023-10-05T21:08:14.039 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:14.037+0000 7f8267b33700 0 -- 172.21.15.167:0/3519920362 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f8248005fb0 msgr2=0x7f8248008460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:14.039 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:14.042 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:14.042 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.33 2023-10-05T21:08:14.123 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:14.121+0000 7f38fdfa5700 0 -- 172.21.15.167:0/546984546 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x55a79a145ee0 msgr2=0x55a79a149fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:14.123 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:14.126 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:14.126 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.34 2023-10-05T21:08:14.209 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:14.207+0000 7f11628f3700 0 -- 172.21.15.167:0/3346008428 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x5581cc2ef4e0 msgr2=0x5581cc300eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:14.210 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:14.213 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:14.213 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.35 2023-10-05T21:08:14.299 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:14.297+0000 7f12a32aa700 0 -- 172.21.15.167:0/4216392464 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f1288006500 msgr2=0x7f1288011eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:14.299 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:14.302 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:14.302 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.36 2023-10-05T21:08:14.386 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:14.384+0000 7f31014d5700 0 -- 172.21.15.167:0/1448629697 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x5576c65df540 msgr2=0x5576c6629100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:14.386 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:14.389 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:14.389 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.37 2023-10-05T21:08:14.502 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:14.501+0000 7fd44848b700 0 -- 172.21.15.167:0/1400417428 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7fd424008740 msgr2=0x7fd42400abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:14.503 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:14.506 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:14.506 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.38 2023-10-05T21:08:14.592 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:14.589+0000 7f74c7665700 0 -- 172.21.15.167:0/4154107523 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x55cc3f3c8a80 msgr2=0x55cc3f3a2560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:14.592 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:14.594 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:14.594 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.39 2023-10-05T21:08:14.681 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:14.679+0000 7f7f6bb6f700 0 -- 172.21.15.167:0/1416989093 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f7f5003a330 msgr2=0x7f7f5003c710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:14.687 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:14.690 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:14.690 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.40 2023-10-05T21:08:14.779 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:14.777+0000 7f0a63a6f700 0 -- 172.21.15.167:0/3973480224 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f0a44005fb0 msgr2=0x7f0a44008460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:14.779 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:14.781 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:14.782 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.41 2023-10-05T21:08:14.867 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:14.865+0000 7f8c9c854700 0 -- 172.21.15.167:0/1004637295 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f8c78008740 msgr2=0x7f8c7800abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:14.867 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:14.871 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:14.871 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.42 2023-10-05T21:08:14.954 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:14.952+0000 7f5784844700 0 -- 172.21.15.167:0/750718090 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f576403a2e0 msgr2=0x7f576403c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:15.155 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:15.158 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:15.158 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.43 2023-10-05T21:08:15.242 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:15.240+0000 7f4bef5f8700 0 -- 172.21.15.167:0/2931262914 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x555b12c4c9b0 msgr2=0x555b12c02c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:15.243 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:15.245 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:15.245 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.44 2023-10-05T21:08:15.329 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:15.327+0000 7f35d8d8d700 0 -- 172.21.15.167:0/3955086815 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f35b4008740 msgr2=0x7f35b400abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-05T21:08:15.335 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:15.338 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:15.338 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.45 2023-10-05T21:08:15.424 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:15.423+0000 7f065de98700 0 -- 172.21.15.167:0/3620539335 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f0644008740 msgr2=0x7f064400abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:15.425 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:15.428 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:15.428 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.46 2023-10-05T21:08:15.513 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:15.511+0000 7f8bacec9700 0 -- 172.21.15.167:0/3377129765 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f8b90008740 msgr2=0x7f8b9000abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:15.514 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:15.517 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:15.517 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.47 2023-10-05T21:08:15.602 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:15.600+0000 7f994cc32700 0 -- 172.21.15.167:0/2250230893 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f9928008740 msgr2=0x7f992800abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:15.603 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:15.606 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:15.606 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.48 2023-10-05T21:08:15.691 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:15.689+0000 7f0112e87700 0 -- 172.21.15.167:0/826237216 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f00f8008740 msgr2=0x7f00f800abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:15.691 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:15.694 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:15.694 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.49 2023-10-05T21:08:15.781 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:15.779+0000 7f70e6397700 0 -- 172.21.15.167:0/4062522316 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f70cc008740 msgr2=0x7f70cc00abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:15.781 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:15.783 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:15.783 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.50 2023-10-05T21:08:15.869 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:15.867+0000 7f4914f5f700 0 -- 172.21.15.167:0/54207040 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f48f00065b0 msgr2=0x7f48f0011ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:15.869 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:15.872 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:15.872 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.51 2023-10-05T21:08:15.961 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:15.958+0000 7f734b112700 0 -- 172.21.15.167:0/2258722811 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x555b3ef0fe80 msgr2=0x555b3eec4820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:15.961 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:15.963 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:15.963 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.52 2023-10-05T21:08:16.047 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:16.045+0000 7f4c94906700 0 -- 172.21.15.167:0/761664807 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x5568b8df45c0 msgr2=0x5568b8df49b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:16.047 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:16.049 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:16.049 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.53 2023-10-05T21:08:16.133 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:16.131+0000 7fea0c2b4700 0 -- 172.21.15.167:0/1044658354 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7fe9e8008740 msgr2=0x7fe9e800abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:16.133 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:16.136 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:16.136 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.54 2023-10-05T21:08:16.224 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:16.222+0000 7f72729a5700 0 -- 172.21.15.167:0/3930742233 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f7258008740 msgr2=0x7f725800abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:16.225 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:16.227 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:16.227 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.55 2023-10-05T21:08:16.312 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:16.310+0000 7fee73db1700 0 -- 172.21.15.167:0/143694966 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7fee54005fb0 msgr2=0x7fee54008460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:16.312 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:16.315 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:16.315 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.56 2023-10-05T21:08:16.403 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:16.401+0000 7f6c60588700 0 -- 172.21.15.167:0/2987096647 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f6c40005fb0 msgr2=0x7f6c40008460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-05T21:08:16.404 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:16.407 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:16.407 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.57 2023-10-05T21:08:16.491 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:16.489+0000 7f649fa41700 0 -- 172.21.15.167:0/910371053 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f647c008740 msgr2=0x7f647c00abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:16.566 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:16.567 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:16.567 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.58 2023-10-05T21:08:16.580 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:16.578+0000 7f2a8ee3b700 0 -- 172.21.15.167:0/3205867908 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f2a6c008740 msgr2=0x7f2a6c00abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:16.580 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:16.583 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:16.583 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.59 2023-10-05T21:08:16.672 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:16.670+0000 7f4290c89700 0 -- 172.21.15.167:0/815741230 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f426c008740 msgr2=0x7f426c00abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-05T21:08:16.673 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:16.676 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:16.676 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.60 2023-10-05T21:08:16.765 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:16.763+0000 7f3e9bedb700 0 -- 172.21.15.167:0/1130520239 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f3e74006720 msgr2=0x7f3e74008ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:16.765 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:16.768 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:16.768 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.61 2023-10-05T21:08:16.857 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:16.855+0000 7f1f5051e700 0 -- 172.21.15.167:0/3299869766 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f1f340087b0 msgr2=0x7f1f34011ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:16.858 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:16.861 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:16.861 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.62 2023-10-05T21:08:16.944 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:16.942+0000 7f0653bd8700 0 -- 172.21.15.167:0/837646039 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f0630006510 msgr2=0x7f0630011ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:16.944 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:16.947 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:16.947 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.63 2023-10-05T21:08:17.034 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:17.032+0000 7f02c15a6700 0 -- 172.21.15.167:0/2591652085 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f029c006500 msgr2=0x7f029c011eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:17.034 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:17.037 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:17.037 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.64 2023-10-05T21:08:17.126 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:17.124+0000 7f0a6f297700 0 -- 172.21.15.167:0/3022377427 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f0a48006720 msgr2=0x7f0a48008ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:17.127 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:17.129 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:17.130 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.65 2023-10-05T21:08:17.217 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:17.215+0000 7fd8837fc700 0 -- 172.21.15.167:0/1000547392 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7fd8680087b0 msgr2=0x7fd868011ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:17.217 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:17.220 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:17.220 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.66 2023-10-05T21:08:17.308 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:17.306+0000 7feb1a4c1700 0 -- 172.21.15.167:0/2569708774 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7feb00008740 msgr2=0x7feb0000abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-05T21:08:17.309 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:17.312 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:17.312 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.67 2023-10-05T21:08:17.394 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:17.392+0000 7f75ad766700 0 -- 172.21.15.167:0/3633297812 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x7f759003a260 msgr2=0x7f759003c640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:17.399 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:17.401 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:17.401 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.68 2023-10-05T21:08:17.489 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:17.487+0000 7faac31c6700 0 -- 172.21.15.167:0/3318622909 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7faaa8008740 msgr2=0x7faaa800abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:17.490 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:17.492 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:17.493 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.69 2023-10-05T21:08:17.579 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:17.577+0000 7f67dcc21700 0 -- 172.21.15.167:0/2600550597 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f67b8008740 msgr2=0x7f67b800abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:17.579 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:17.582 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:17.582 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.70 2023-10-05T21:08:17.673 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:17.671+0000 7fc085daf700 0 -- 172.21.15.167:0/3866874349 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7fc064008740 msgr2=0x7fc06400abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:17.673 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:17.676 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:17.676 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.71 2023-10-05T21:08:17.827 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:17.825+0000 7f87c5bd3700 0 -- 172.21.15.167:0/2568811133 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f87a803a3b0 msgr2=0x7f87a803c790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-05T21:08:17.834 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:17.836 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:17.836 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.72 2023-10-05T21:08:17.921 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:17.919+0000 7fda4f47c700 0 -- 172.21.15.167:0/4254391238 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7fda2c006500 msgr2=0x7fda2c011eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-05T21:08:17.922 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:17.925 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:17.925 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.73 2023-10-05T21:08:18.012 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:18.010+0000 7fdcc95ba700 0 -- 172.21.15.167:0/2556806228 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7fdca4008740 msgr2=0x7fdca400abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:18.013 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:18.015 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:18.016 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.74 2023-10-05T21:08:18.107 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:18.105+0000 7f67e74a5700 0 -- 172.21.15.167:0/3907446431 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f67c4008740 msgr2=0x7f67c400abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:18.108 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:18.110 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:18.110 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.75 2023-10-05T21:08:18.197 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:18.195+0000 7f9da01b2700 0 -- 172.21.15.167:0/2449771886 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f9d7c008740 msgr2=0x7f9d7c00abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:18.198 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:18.200 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:18.200 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.76 2023-10-05T21:08:18.289 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:18.287+0000 7f1e984cb700 0 -- 172.21.15.167:0/3549913840 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f1e74008740 msgr2=0x7f1e7400abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:18.289 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:18.292 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:18.292 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.77 2023-10-05T21:08:18.379 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:18.377+0000 7f861bbeb700 0 -- 172.21.15.167:0/3401452384 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x55e480a2ccd0 msgr2=0x55e480a044d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:18.380 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:18.383 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:18.383 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.78 2023-10-05T21:08:18.528 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:18.526+0000 7f6d051c5700 0 -- 172.21.15.167:0/358017884 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f6ce0006500 msgr2=0x7f6ce0015780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-05T21:08:18.529 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:18.532 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:18.532 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.79 2023-10-05T21:08:18.634 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:18.632+0000 7f6659a8b700 0 -- 172.21.15.167:0/1489643912 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f6638006500 msgr2=0x7f6638011eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:18.635 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:18.637 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:18.637 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.80 2023-10-05T21:08:18.750 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:18.748+0000 7fd0d0284700 0 -- 172.21.15.167:0/4080431490 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7fd0ac008740 msgr2=0x7fd0ac00abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:18.751 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:18.753 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:18.754 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.81 2023-10-05T21:08:18.844 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:18.842+0000 7fda5b6ef700 0 -- 172.21.15.167:0/1540549547 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7fda38008740 msgr2=0x7fda3800abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:18.844 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:18.847 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:18.847 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.82 2023-10-05T21:08:18.933 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:18.931+0000 7f132f524700 0 -- 172.21.15.167:0/270649027 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f1310006a00 msgr2=0x7f1310008de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:18.934 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:18.936 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:18.936 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.83 2023-10-05T21:08:19.028 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:19.026+0000 7f3089ac1700 0 -- 172.21.15.167:0/553814591 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x5635418fec60 msgr2=0x56354190c240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:19.028 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:19.030 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:19.031 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.84 2023-10-05T21:08:19.127 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:19.125+0000 7f3b00bb1700 0 -- 172.21.15.167:0/3770041761 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f3ae0005fb0 msgr2=0x7f3ae0008460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:19.127 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:19.130 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:19.130 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.85 2023-10-05T21:08:19.220 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:19.218+0000 7fbd20a57700 0 -- 172.21.15.167:0/2766133582 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x55572b67db80 msgr2=0x55572b6555c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:19.220 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:19.223 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:19.223 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.86 2023-10-05T21:08:19.311 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:19.309+0000 7fe305705700 0 -- 172.21.15.167:0/977697597 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7fe2e00088b0 msgr2=0x7fe2e000ad30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:19.311 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:19.314 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:19.314 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.87 2023-10-05T21:08:19.401 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:19.399+0000 7f3005dd6700 0 -- 172.21.15.167:0/625809140 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f2fec008740 msgr2=0x7f2fec00abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:19.402 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:19.405 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:19.405 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.88 2023-10-05T21:08:19.499 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:19.496+0000 7f0dedc40700 0 -- 172.21.15.167:0/580271581 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x7f0dd003a330 msgr2=0x7f0dd003c710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:19.707 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:19.709 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:19.709 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.89 2023-10-05T21:08:19.794 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:19.792+0000 7f5790760700 0 -- 172.21.15.167:0/3411976831 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f576c008740 msgr2=0x7f576c00abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-05T21:08:19.796 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:19.799 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:19.799 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.90 2023-10-05T21:08:19.890 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:19.888+0000 7f58b5696700 0 -- 172.21.15.167:0/1095782524 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x55f8489fad50 msgr2=0x55f848a1cf80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:19.891 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:19.893 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:19.893 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.91 2023-10-05T21:08:19.981 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:19.980+0000 7fa79c57d700 0 -- 172.21.15.167:0/1974238015 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7fa778008740 msgr2=0x7fa77800abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:19.982 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:19.985 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:19.985 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.92 2023-10-05T21:08:20.073 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:20.071+0000 7f41e47f9700 0 -- 172.21.15.167:0/2956696746 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x5612b09ebcc0 msgr2=0x5612b09d7970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:20.073 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:20.076 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:20.076 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.93 2023-10-05T21:08:20.164 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:20.162+0000 7f6c67762700 0 -- 172.21.15.167:0/977976493 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x55f6a44bb120 msgr2=0x55f6a44bd060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:20.165 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:20.167 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:20.168 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.94 2023-10-05T21:08:20.258 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:20.256+0000 7fca1b5f6700 0 -- 172.21.15.167:0/1378026482 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7fca00008740 msgr2=0x7fca0000abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:20.259 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:20.262 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:20.262 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.95 2023-10-05T21:08:20.351 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:20.350+0000 7f40f8339700 0 -- 172.21.15.167:0/2148106620 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f40dc03a330 msgr2=0x7f40dc03c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:20.553 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:20.556 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:20.556 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.96 2023-10-05T21:08:20.643 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:20.641+0000 7f8a91851700 0 -- 172.21.15.167:0/3221305472 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x562e84af9370 msgr2=0x562e84b21270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:20.643 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:20.645 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:20.646 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.97 2023-10-05T21:08:20.733 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:20.731+0000 7fb706aec700 0 -- 172.21.15.167:0/4111651835 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7fb6e4008740 msgr2=0x7fb6e400abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-05T21:08:20.739 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:20.742 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:20.742 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.98 2023-10-05T21:08:20.829 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:20.827+0000 7fdb96fc3700 0 -- 172.21.15.167:0/1793971000 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x5653d030d3e0 msgr2=0x5653d032f240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:20.830 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:20.832 INFO:tasks.workunit.client.0.smithi167.stderr:+ for i in $(seq -w 00 99) 2023-10-05T21:08:20.833 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm image.99 2023-10-05T21:08:20.915 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:20.913+0000 7fe8b7523700 0 -- 172.21.15.167:0/529670019 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7fe894008740 msgr2=0x7fe89400abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-05T21:08:20.921 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:20.924 INFO:tasks.workunit.client.0.smithi167.stderr:+ test_remove 2023-10-05T21:08:20.924 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'testing remove...' 2023-10-05T21:08:20.924 INFO:tasks.workunit.client.0.smithi167.stdout:testing remove... 2023-10-05T21:08:20.925 INFO:tasks.workunit.client.0.smithi167.stderr:+ remove_images 2023-10-05T21:08:20.925 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:20.999 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:21.074 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:21.147 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:21.221 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:21.307 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:21.380 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:21.453 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:21.527 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:21.601 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:21.674 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:21.749 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:21.822 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:21.897 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:21.971 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:22.044 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:22.120 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:22.194 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:22.269 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd remove NOT_EXIST 2023-10-05T21:08:22.307 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 0% complete...failed. 2023-10-05T21:08:22.307 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: delete error: (2) No such file or directory 2023-10-05T21:08:22.309 INFO:tasks.workunit.client.0.smithi167.stderr:+ true 2023-10-05T21:08:22.310 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 1 -s 1 test1 2023-10-05T21:08:22.333 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: image format 1 is deprecated 2023-10-05T21:08:22.338 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:22.337+0000 7efcf31a5540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:22.345 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test1 2023-10-05T21:08:22.383 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:22.386 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls 2023-10-05T21:08:22.386 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:08:22.387 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '^0$' 2023-10-05T21:08:22.414 INFO:tasks.workunit.client.0.smithi167.stdout:0 2023-10-05T21:08:22.414 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 2 -s 1 test2 2023-10-05T21:08:22.457 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test2 2023-10-05T21:08:22.542 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:22.541+0000 7fbac21c9700 0 -- 172.21.15.167:0/562356057 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7fba9c006790 msgr2=0x7fba9c008c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:22.543 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:22.546 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls 2023-10-05T21:08:22.546 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:08:22.547 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '^0$' 2023-10-05T21:08:22.574 INFO:tasks.workunit.client.0.smithi167.stdout:0 2023-10-05T21:08:22.574 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 1 -s 1 test1 2023-10-05T21:08:22.597 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: image format 1 is deprecated 2023-10-05T21:08:22.603 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:22.601+0000 7f9f02d86540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:22.609 INFO:tasks.workunit.client.0.smithi167.stderr:+ rados rm -p rbd test1.rbd 2023-10-05T21:08:22.639 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test1 2023-10-05T21:08:22.670 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:22.673 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls 2023-10-05T21:08:22.673 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:08:22.673 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '^0$' 2023-10-05T21:08:22.700 INFO:tasks.workunit.client.0.smithi167.stdout:0 2023-10-05T21:08:22.700 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 1 -eq 0 ']' 2023-10-05T21:08:22.700 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 2 -s 1 test2 2023-10-05T21:08:22.741 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap create test2@snap 2023-10-05T21:08:23.674 INFO:tasks.workunit.client.0.smithi167.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-05T21:08:23.683 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap protect test2@snap 2023-10-05T21:08:23.734 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd clone test2@snap clone --rbd-default-clone-format 1 2023-10-05T21:08:23.798 INFO:tasks.workunit.client.0.smithi167.stderr:+ rados -p rbd rm rbd_children 2023-10-05T21:08:23.828 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm clone 2023-10-05T21:08:23.923 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:23.921+0000 7f8a184a9700 0 -- 172.21.15.167:0/2624839816 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f89fc006540 msgr2=0x7f89fc011e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:23.924 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:23.927 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls 2023-10-05T21:08:23.927 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep clone 2023-10-05T21:08:23.927 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:08:23.927 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '^0$' 2023-10-05T21:08:23.954 INFO:tasks.workunit.client.0.smithi167.stdout:0 2023-10-05T21:08:23.954 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap unprotect test2@snap 2023-10-05T21:08:23.998 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap rm test2@snap 2023-10-05T21:08:24.674 INFO:tasks.workunit.client.0.smithi167.stderr: Removing snap: 100% complete...done. 2023-10-05T21:08:24.684 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test2 2023-10-05T21:08:24.787 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:24.785+0000 7f12a1cb8700 0 -- 172.21.15.167:0/3328917306 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f1288008740 msgr2=0x7f128800abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:24.788 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:24.791 INFO:tasks.workunit.client.0.smithi167.stderr:+ test_migration 2023-10-05T21:08:24.791 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'testing migration...' 2023-10-05T21:08:24.791 INFO:tasks.workunit.client.0.smithi167.stdout:testing migration... 2023-10-05T21:08:24.792 INFO:tasks.workunit.client.0.smithi167.stderr:+ remove_images 2023-10-05T21:08:24.792 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:24.867 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:24.941 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:25.016 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:25.090 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:25.167 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:25.241 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:25.317 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:25.394 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:25.470 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:25.545 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:25.620 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:25.695 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:25.774 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:25.850 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:25.927 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:26.013 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:26.088 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:26.163 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph osd pool create rbd2 8 2023-10-05T21:08:26.794 INFO:tasks.workunit.client.0.smithi167.stderr:pool 'rbd2' already exists 2023-10-05T21:08:26.804 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd pool init rbd2 2023-10-05T21:08:29.686 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 1 -s 128M test1 2023-10-05T21:08:29.712 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: image format 1 is deprecated 2023-10-05T21:08:29.720 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:29.719+0000 7f4462eba540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:29.728 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info test1 2023-10-05T21:08:29.728 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'format: 1' 2023-10-05T21:08:29.762 INFO:tasks.workunit.client.0.smithi167.stdout: format: 1 2023-10-05T21:08:29.763 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration prepare test1 --image-format 2 2023-10-05T21:08:29.836 INFO:tasks.workunit.client.0.smithi167.stderr:++ get_migration_state test1 2023-10-05T21:08:29.837 INFO:tasks.workunit.client.0.smithi167.stderr:++ local image=test1 2023-10-05T21:08:29.837 INFO:tasks.workunit.client.0.smithi167.stderr:++ rbd --format xml status test1 2023-10-05T21:08:29.837 INFO:tasks.workunit.client.0.smithi167.stderr:++ xmlstarlet sel -t -v //status/migration/state 2023-10-05T21:08:29.908 INFO:tasks.workunit.client.0.smithi167.stderr:+ test prepared = prepared 2023-10-05T21:08:29.908 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info test1 2023-10-05T21:08:29.908 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'format: 2' 2023-10-05T21:08:29.945 INFO:tasks.workunit.client.0.smithi167.stdout: format: 2 2023-10-05T21:08:29.945 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test1 2023-10-05T21:08:29.980 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:29.979+0000 7effa039a540 -1 librbd::image::PreRemoveRequest: 0x55d0fb5ec8a0 validate_image_removal: image in migration state - not removing 2023-10-05T21:08:29.981 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 0% complete...failed. 2023-10-05T21:08:29.981 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: error: image still has watchers 2023-10-05T21:08:29.981 INFO:tasks.workunit.client.0.smithi167.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2023-10-05T21:08:29.984 INFO:tasks.workunit.client.0.smithi167.stderr:+ true 2023-10-05T21:08:29.984 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration execute test1 2023-10-05T21:08:30.094 INFO:tasks.workunit.client.0.smithi167.stderr: Image migration: 3% complete... Image migration: 6% complete... Image migration: 9% complete... Image migration: 12% complete... Image migration: 15% complete... Image migration: 18% complete... Image migration: 21% complete... Image migration: 25% complete... Image migration: 28% complete... Image migration: 31% complete... Image migration: 34% complete... Image migration: 37% complete... Image migration: 40% complete... Image migration: 43% complete... Image migration: 46% complete... Image migration: 50% complete... Image migration: 53% complete... Image migration: 56% complete... Image migration: 59% complete... Image migration: 62% complete... Image migration: 65% complete... Image migration: 68% complete... Image migration: 71% complete... Image migration: 75% complete... Image migration: 78% complete... Image migration: 81% complete... Image migration: 84% complete... Image migration: 87% complete... Image migration: 90% complete... Image migration: 93% complete... Image migration: 96% complete...2023-10-05T21:08:30.093+0000 7f12f472c700 0 -- 172.21.15.167:0/241312224 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x55fb474c15b0 msgr2=0x55fb4751f870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:30.113 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:30.112+0000 7f12f3f2b700 0 -- 172.21.15.167:0/241312224 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x7f12d0008740 msgr2=0x7f12d000abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:30.128 INFO:tasks.workunit.client.0.smithi167.stderr: Image migration: 100% complete...done. 2023-10-05T21:08:30.131 INFO:tasks.workunit.client.0.smithi167.stderr:++ get_migration_state test1 2023-10-05T21:08:30.132 INFO:tasks.workunit.client.0.smithi167.stderr:++ local image=test1 2023-10-05T21:08:30.132 INFO:tasks.workunit.client.0.smithi167.stderr:++ rbd --format xml status test1 2023-10-05T21:08:30.132 INFO:tasks.workunit.client.0.smithi167.stderr:++ xmlstarlet sel -t -v //status/migration/state 2023-10-05T21:08:30.194 INFO:tasks.workunit.client.0.smithi167.stderr:+ test executed = executed 2023-10-05T21:08:30.194 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration commit test1 2023-10-05T21:08:30.282 INFO:tasks.workunit.client.0.smithi167.stderr: Commit image migration: 3% complete... Commit image migration: 6% complete... Commit image migration: 9% complete... Commit image migration: 12% complete... Commit image migration: 15% complete... Commit image migration: 18% complete... Commit image migration: 21% complete... Commit image migration: 25% complete... Commit image migration: 28% complete... Commit image migration: 31% complete... Commit image migration: 34% complete... Commit image migration: 37% complete... Commit image migration: 40% complete... Commit image migration: 43% complete... 2023-10-05T21:08:30.280+0000 7fb27fc45700 0 -- 172.21.15.167:0/4099332086 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x560f2adaa920 msgr2=0x560f2ae08df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:30.290 INFO:tasks.workunit.client.0.smithi167.stderr:Commit image migration: 46% complete... Commit image migration: 50% complete... Commit image migration: 53% complete... Commit image migration: 56% complete... Commit image migration: 59% complete... Commit image migration: 62% complete... Commit image migration: 65% complete... Commit image migration: 68% complete... Commit image migration: 71% complete... Commit image migration: 75% complete... Commit image migration: 78% complete... Commit image migration: 81% complete... Commit image migration: 84% complete... Commit image migration: 87% complete... Commit image migration: 90% complete... Commit image migration: 93% complete... Commit image migration: 96% complete...2023-10-05T21:08:30.289+0000 7fb27f444700 0 -- 172.21.15.167:0/4099332086 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x7fb264008740 msgr2=0x7fb26400abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:30.306 INFO:tasks.workunit.client.0.smithi167.stderr: Commit image migration: 100% complete...done. 2023-10-05T21:08:30.309 INFO:tasks.workunit.client.0.smithi167.stderr:+ get_migration_state test1 2023-10-05T21:08:30.310 INFO:tasks.workunit.client.0.smithi167.stderr:+ local image=test1 2023-10-05T21:08:30.310 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd --format xml status test1 2023-10-05T21:08:30.310 INFO:tasks.workunit.client.0.smithi167.stderr:+ xmlstarlet sel -t -v //status/migration/state 2023-10-05T21:08:30.345 INFO:tasks.workunit.client.0.smithi167.stderr:+ true 2023-10-05T21:08:30.345 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info test1 2023-10-05T21:08:30.346 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'features: .*layering' 2023-10-05T21:08:30.380 INFO:tasks.workunit.client.0.smithi167.stderr:+ true 2023-10-05T21:08:30.380 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration prepare test1 --image-feature layering,exclusive-lock,object-map,fast-diff,deep-flatten 2023-10-05T21:08:30.476 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info test1 2023-10-05T21:08:30.476 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'features: .*layering' 2023-10-05T21:08:30.516 INFO:tasks.workunit.client.0.smithi167.stdout: features: layering, exclusive-lock, object-map, fast-diff, deep-flatten, migrating 2023-10-05T21:08:30.516 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration execute test1 2023-10-05T21:08:30.595 INFO:tasks.workunit.client.0.smithi167.stderr: Image migration: 3% complete... Image migration: 6% complete... Image migration: 9% complete... Image migration: 12% complete... Image migration: 15% complete... Image migration: 18% complete... Image migration: 21% complete... Image migration: 25% complete... Image migration: 28% complete... Image migration: 31% complete... Image migration: 34% complete... Image migration: 37% complete... Image migration: 40% complete... Image migration: 43% complete... Image migration: 46% complete... Image migration: 50% complete... Image migration: 53% complete... Image migration: 56% complete... Image migration: 59% complete... Image migration: 62% complete... Image migration: 65% complete... Image migration: 68% complete... Image migration: 71% complete... Image migration: 75% complete... Image migration: 78% complete... Image migration: 81% complete... Image migration: 84% complete... Image migration: 87% complete... Image migration: 90% complete... Image migration: 93% complete... Image migration: 96% complete...2023-10-05T21:08:30.594+0000 7fd2a594c700 0 -- 172.21.15.167:0/1647165240 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x7fd27c004b40 msgr2=0x7fd27c006f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:30.614 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:30.613+0000 7fd2a614d700 0 -- 172.21.15.167:0/1647165240 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x55635d4285b0 msgr2=0x55635d487cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:30.616 INFO:tasks.workunit.client.0.smithi167.stderr: Image migration: 100% complete...done. 2023-10-05T21:08:30.619 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration commit test1 2023-10-05T21:08:30.699 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:30.698+0000 7fd8ab115700 0 -- 172.21.15.167:0/1846038195 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x565044161920 msgr2=0x5650441bff60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:30.710 INFO:tasks.workunit.client.0.smithi167.stderr: Commit image migration: 3% complete... Commit image migration: 6% complete... Commit image migration: 9% complete... Commit image migration: 12% complete... Commit image migration: 15% complete... Commit image migration: 18% complete... Commit image migration: 21% complete... Commit image migration: 25% complete... Commit image migration: 28% complete... Commit image migration: 31% complete... Commit image migration: 34% complete... Commit image migration: 37% complete... Commit image migration: 40% complete... Commit image migration: 43% complete... Commit image migration: 46% complete... Commit image migration: 50% complete... Commit image migration: 53% complete...2023-10-05T21:08:30.709+0000 7fd8aa914700 0 -- 172.21.15.167:0/1846038195 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x7fd884004be0 msgr2=0x7fd884006fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:30.744 INFO:tasks.workunit.client.0.smithi167.stderr: Commit image migration: 56% complete... Commit image migration: 59% complete... Commit image migration: 62% complete... Commit image migration: 65% complete... Commit image migration: 68% complete... Commit image migration: 71% complete... Commit image migration: 75% complete... Commit image migration: 78% complete... Commit image migration: 81% complete... Commit image migration: 84% complete... Commit image migration: 87% complete... Commit image migration: 90% complete... Commit image migration: 93% complete... Commit image migration: 96% complete... Commit image migration: 100% complete...done. 2023-10-05T21:08:30.746 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration prepare test1 rbd2/test1 2023-10-05T21:08:30.837 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:30.837+0000 7fd6d636b700 0 -- 172.21.15.167:0/2716227304 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x7fd6b0004be0 msgr2=0x7fd6b0006fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:30.844 INFO:tasks.workunit.client.0.smithi167.stderr:++ get_migration_state rbd2/test1 2023-10-05T21:08:30.844 INFO:tasks.workunit.client.0.smithi167.stderr:++ local image=rbd2/test1 2023-10-05T21:08:30.844 INFO:tasks.workunit.client.0.smithi167.stderr:++ rbd --format xml status rbd2/test1 2023-10-05T21:08:30.844 INFO:tasks.workunit.client.0.smithi167.stderr:++ xmlstarlet sel -t -v //status/migration/state 2023-10-05T21:08:30.916 INFO:tasks.workunit.client.0.smithi167.stderr:+ test prepared = prepared 2023-10-05T21:08:30.916 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls 2023-10-05T21:08:30.916 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:08:30.917 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '^0$' 2023-10-05T21:08:30.943 INFO:tasks.workunit.client.0.smithi167.stdout:0 2023-10-05T21:08:30.943 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd -p rbd2 ls 2023-10-05T21:08:30.943 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep test1 2023-10-05T21:08:30.972 INFO:tasks.workunit.client.0.smithi167.stdout:test1 2023-10-05T21:08:30.972 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration execute test1 2023-10-05T21:08:31.056 INFO:tasks.workunit.client.0.smithi167.stderr: Image migration: 3% complete... Image migration: 6% complete... Image migration: 9% complete... Image migration: 12% complete... Image migration: 15% complete... Image migration: 18% complete... Image migration: 21% complete... Image migration: 25% complete... Image migration: 28% complete... Image migration: 31% complete... Image migration: 34% complete... Image migration: 37% complete... Image migration: 40% complete... Image migration: 43% complete... Image migration: 46% complete... Image migration: 50% complete... Image migration: 53% complete... Image migration: 56% complete... Image migration: 59% complete... Image migration: 62% complete... Image migration: 65% complete... Image migration: 68% complete... Image migration: 71% complete... Image migration: 75% complete... Image migration: 78% complete... Image migration: 81% complete... Image migration: 84% complete... Image migration: 87% complete... Image migration: 90% complete... Image migration: 93% complete... Image migration: 96% complete...2023-10-05T21:08:31.055+0000 7fc81b5c9700 0 -- 172.21.15.167:0/821806834 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x56139525a660 msgr2=0x56139525ca40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:31.063 INFO:tasks.workunit.client.0.smithi167.stderr: Image migration: 100% complete...done. 2023-10-05T21:08:31.067 INFO:tasks.workunit.client.0.smithi167.stderr:++ get_migration_state rbd2/test1 2023-10-05T21:08:31.067 INFO:tasks.workunit.client.0.smithi167.stderr:++ local image=rbd2/test1 2023-10-05T21:08:31.067 INFO:tasks.workunit.client.0.smithi167.stderr:++ rbd --format xml status rbd2/test1 2023-10-05T21:08:31.067 INFO:tasks.workunit.client.0.smithi167.stderr:++ xmlstarlet sel -t -v //status/migration/state 2023-10-05T21:08:31.132 INFO:tasks.workunit.client.0.smithi167.stderr:+ test executed = executed 2023-10-05T21:08:31.132 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm rbd2/test1 2023-10-05T21:08:31.170 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:31.170+0000 7f7deeffd700 -1 librbd::image::PreRemoveRequest: 0x55e719b03590 validate_image_removal: image in migration state - not removing 2023-10-05T21:08:31.179 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 0% complete...failed. 2023-10-05T21:08:31.179 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: error: image still has watchers 2023-10-05T21:08:31.179 INFO:tasks.workunit.client.0.smithi167.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2023-10-05T21:08:31.181 INFO:tasks.workunit.client.0.smithi167.stderr:+ true 2023-10-05T21:08:31.181 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration commit test1 2023-10-05T21:08:31.263 INFO:tasks.workunit.client.0.smithi167.stderr: Commit image migration: 3% complete... Commit image migration: 6% complete... Commit image migration: 9% complete... Commit image migration: 12% complete... Commit image migration: 15% complete... Commit image migration: 18% complete... Commit image migration: 21% complete... Commit image migration: 25% complete... Commit image migration: 28% complete... Commit image migration: 31% complete... Commit image migration: 34% complete... Commit image migration: 37% complete... Commit image migration: 40% complete... Commit image migration: 43% complete... Commit image migration: 46% complete... Commit image migration: 50% complete... Commit image migration: 53% complete... Commit image migration: 56% complete... Commit image migration: 59% complete... Commit image migration: 62% complete... Commit image migration: 65% complete... Commit image migration: 68% complete... Commit image migration: 71% complete... Commit image migration: 75% complete... Commit image migration: 78% complete... Commit image migration: 81% complete... Commit image migration: 84% complete... Commit image migration: 87% complete... Commit image migration: 90% complete... Commit image migration: 93% complete... Commit image migration: 96% complete...2023-10-05T21:08:31.263+0000 7fac8c3e1700 0 -- 172.21.15.167:0/1827945600 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x55a905893cb0 msgr2=0x55a90598adb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:31.283 INFO:tasks.workunit.client.0.smithi167.stderr: Commit image migration: 100% complete...done. 2023-10-05T21:08:31.287 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd namespace create rbd2/ns1 2023-10-05T21:08:31.317 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd namespace create rbd2/ns2 2023-10-05T21:08:31.347 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration prepare rbd2/test1 rbd2/ns1/test1 2023-10-05T21:08:31.426 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:31.425+0000 7f669ab83700 0 -- 172.21.15.167:0/456309101 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x5617c716f650 msgr2=0x5617c7211750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:31.432 INFO:tasks.workunit.client.0.smithi167.stderr:++ get_migration_state rbd2/ns1/test1 2023-10-05T21:08:31.432 INFO:tasks.workunit.client.0.smithi167.stderr:++ local image=rbd2/ns1/test1 2023-10-05T21:08:31.432 INFO:tasks.workunit.client.0.smithi167.stderr:++ rbd --format xml status rbd2/ns1/test1 2023-10-05T21:08:31.432 INFO:tasks.workunit.client.0.smithi167.stderr:++ xmlstarlet sel -t -v //status/migration/state 2023-10-05T21:08:31.504 INFO:tasks.workunit.client.0.smithi167.stderr:+ test prepared = prepared 2023-10-05T21:08:31.504 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration execute rbd2/test1 2023-10-05T21:08:31.573 INFO:tasks.workunit.client.0.smithi167.stderr: Image migration: 3% complete... Image migration: 6% complete... Image migration: 9% complete... Image migration: 12% complete... Image migration: 15% complete... Image migration: 18% complete... Image migration: 21% complete... Image migration: 25% complete... Image migration: 28% complete... Image migration: 31% complete... Image migration: 34% complete... Image migration: 37% complete... Image migration: 40% complete... Image migration: 43% complete... Image migration: 46% complete... Image migration: 50% complete... Image migration: 53% complete... Image migration: 56% complete... Image migration: 59% complete... Image migration: 62% complete... Image migration: 65% complete... Image migration: 68% complete... Image migration: 71% complete... Image migration: 75% complete... Image migration: 78% complete... Image migration: 81% complete... Image migration: 84% complete... Image migration: 87% complete... Image migration: 90% complete... Image migration: 93% complete... Image migration: 96% complete...2023-10-05T21:08:31.572+0000 7f5ad9e77700 0 -- 172.21.15.167:0/904906908 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x555ac7dbe270 msgr2=0x555ac7eb45f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:31.576 INFO:tasks.workunit.client.0.smithi167.stderr: Image migration: 100% complete...done. 2023-10-05T21:08:31.580 INFO:tasks.workunit.client.0.smithi167.stderr:++ get_migration_state rbd2/ns1/test1 2023-10-05T21:08:31.580 INFO:tasks.workunit.client.0.smithi167.stderr:++ local image=rbd2/ns1/test1 2023-10-05T21:08:31.580 INFO:tasks.workunit.client.0.smithi167.stderr:++ rbd --format xml status rbd2/ns1/test1 2023-10-05T21:08:31.580 INFO:tasks.workunit.client.0.smithi167.stderr:++ xmlstarlet sel -t -v //status/migration/state 2023-10-05T21:08:31.643 INFO:tasks.workunit.client.0.smithi167.stderr:+ test executed = executed 2023-10-05T21:08:31.643 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration commit rbd2/test1 2023-10-05T21:08:31.718 INFO:tasks.workunit.client.0.smithi167.stderr: Commit image migration: 3% complete... Commit image migration: 6% complete... Commit image migration: 9% complete... Commit image migration: 12% complete... Commit image migration: 15% complete... Commit image migration: 18% complete... Commit image migration: 21% complete... Commit image migration: 25% complete... Commit image migration: 28% complete... Commit image migration: 31% complete... Commit image migration: 34% complete... Commit image migration: 37% complete... Commit image migration: 40% complete... Commit image migration: 43% complete... Commit image migration: 46% complete... Commit image migration: 50% complete... Commit image migration: 53% complete... Commit image migration: 56% complete... Commit image migration: 59% complete... Commit image migration: 62% complete... Commit image migration: 65% complete... Commit image migration: 68% complete... Commit image migration: 71% complete... Commit image migration: 75% complete... Commit image migration: 78% complete... Commit image migration: 81% complete... Commit image migration: 84% complete... Commit image migration: 87% complete... Commit image migration: 90% complete... Commit image migration: 93% complete... Commit image migration: 96% complete...2023-10-05T21:08:31.717+0000 7efe537f2700 0 -- 172.21.15.167:0/3602579256 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x55a8504d9c30 msgr2=0x55a8504dc010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:31.730 INFO:tasks.workunit.client.0.smithi167.stderr: Commit image migration: 100% complete...done. 2023-10-05T21:08:31.733 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration prepare rbd2/ns1/test1 rbd2/ns2/test1 2023-10-05T21:08:31.812 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:31.811+0000 7f707a5c6700 0 -- 172.21.15.167:0/2534109177 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x55fb13acded0 msgr2=0x55fb13b70780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:31.818 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration execute rbd2/ns2/test1 2023-10-05T21:08:31.883 INFO:tasks.workunit.client.0.smithi167.stderr: Image migration: 3% complete... Image migration: 6% complete... Image migration: 9% complete... Image migration: 12% complete... Image migration: 15% complete... Image migration: 18% complete... Image migration: 21% complete... Image migration: 25% complete... Image migration: 28% complete... Image migration: 31% complete... Image migration: 34% complete... Image migration: 37% complete... Image migration: 40% complete... Image migration: 43% complete... Image migration: 46% complete... Image migration: 50% complete... Image migration: 53% complete... Image migration: 56% complete... Image migration: 59% complete... Image migration: 62% complete... Image migration: 65% complete... Image migration: 68% complete... Image migration: 71% complete... Image migration: 75% complete... Image migration: 78% complete... Image migration: 81% complete... Image migration: 84% complete... Image migration: 87% complete... Image migration: 90% complete... Image migration: 93% complete... Image migration: 96% complete...2023-10-05T21:08:31.882+0000 7f250387e700 0 -- 172.21.15.167:0/1595967777 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x55e6fb0864e0 msgr2=0x55e6fb0888c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:31.890 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:31.890+0000 7f250387e700 0 -- 172.21.15.167:0/1595967777 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f24e803aab0 msgr2=0x7f24e803ced0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:31.892 INFO:tasks.workunit.client.0.smithi167.stderr: Image migration: 100% complete...done. 2023-10-05T21:08:31.895 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration commit rbd2/ns2/test1 2023-10-05T21:08:31.970 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:31.969+0000 7f5599223700 0 -- 172.21.15.167:0/3675471808 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x5614b041f960 msgr2=0x5614b0421d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:31.977 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:31.976+0000 7f5599223700 0 -- 172.21.15.167:0/3675471808 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x5614b03be920 msgr2=0x7f557c03fc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:31.994 INFO:tasks.workunit.client.0.smithi167.stderr: Commit image migration: 3% complete... Commit image migration: 6% complete... Commit image migration: 9% complete... Commit image migration: 12% complete... Commit image migration: 15% complete... Commit image migration: 18% complete... Commit image migration: 21% complete... Commit image migration: 25% complete... Commit image migration: 28% complete... Commit image migration: 31% complete... Commit image migration: 34% complete... Commit image migration: 37% complete... Commit image migration: 40% complete... Commit image migration: 43% complete... Commit image migration: 46% complete... Commit image migration: 50% complete... Commit image migration: 53% complete... Commit image migration: 56% complete... Commit image migration: 59% complete... Commit image migration: 62% complete... Commit image migration: 65% complete... Commit image migration: 68% complete... Commit image migration: 71% complete... Commit image migration: 75% complete... Commit image migration: 78% complete... Commit image migration: 81% complete... Commit image migration: 84% complete... Commit image migration: 87% complete... Commit image migration: 90% complete... Commit image migration: 93% complete... Commit image migration: 96% complete... Commit image migration: 100% complete...done. 2023-10-05T21:08:31.997 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create -s 128M test1 2023-10-05T21:08:32.025 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:32.024+0000 7fd1e2fb8540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:32.031 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration prepare test1 --data-pool rbd2 2023-10-05T21:08:32.114 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info test1 2023-10-05T21:08:32.114 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'data_pool: rbd2' 2023-10-05T21:08:32.150 INFO:tasks.workunit.client.0.smithi167.stdout: data_pool: rbd2 2023-10-05T21:08:32.150 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration execute test1 2023-10-05T21:08:32.226 INFO:tasks.workunit.client.0.smithi167.stderr: Image migration: 3% complete... Image migration: 6% complete... Image migration: 9% complete... Image migration: 12% complete... Image migration: 15% complete... Image migration: 18% complete... Image migration: 21% complete... Image migration: 25% complete... Image migration: 28% complete... Image migration: 31% complete... Image migration: 34% complete... Image migration: 37% complete... Image migration: 40% complete... Image migration: 43% complete... Image migration: 46% complete... Image migration: 50% complete... Image migration: 53% complete... Image migration: 56% complete... Image migration: 59% complete... Image migration: 62% complete... Image migration: 65% complete... Image migration: 68% complete... Image migration: 71% complete... Image migration: 75% complete... Image migration: 78% complete... Image migration: 81% complete... Image migration: 84% complete... Image migration: 87% complete...2023-10-05T21:08:32.225+0000 7f7b21f74700 0 -- 172.21.15.167:0/1537571612 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f7b0403aab0 msgr2=0x7f7b0403ced0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:32.236 INFO:tasks.workunit.client.0.smithi167.stderr: Image migration: 90% complete... Image migration: 93% complete... Image migration: 96% complete...2023-10-05T21:08:32.236+0000 7f7b21f74700 0 -- 172.21.15.167:0/1537571612 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x560c7f5228c0 msgr2=0x7f7b0403fc10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:32.258 INFO:tasks.workunit.client.0.smithi167.stderr: Image migration: 100% complete...done. 2023-10-05T21:08:32.260 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration commit test1 2023-10-05T21:08:32.353 INFO:tasks.workunit.client.0.smithi167.stderr: Commit image migration: 3% complete... Commit image migration: 6% complete... Commit image migration: 9% complete... Commit image migration: 12% complete... Commit image migration: 15% complete... Commit image migration: 18% complete... Commit image migration: 21% complete... Commit image migration: 25% complete... Commit image migration: 28% complete... Commit image migration: 31% complete... Commit image migration: 34% complete... Commit image migration: 37% complete... Commit image migration: 40% complete... Commit image migration: 43% complete...2023-10-05T21:08:32.352+0000 7f85d89cb700 0 -- 172.21.15.167:0/2512068676 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x7f85b8004b90 msgr2=0x7f85b8006f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:32.364 INFO:tasks.workunit.client.0.smithi167.stderr: Commit image migration: 46% complete... Commit image migration: 50% complete... Commit image migration: 53% complete... Commit image migration: 56% complete... Commit image migration: 59% complete... Commit image migration: 62% complete... Commit image migration: 65% complete... Commit image migration: 68% complete... Commit image migration: 71% complete... Commit image migration: 75% complete... Commit image migration: 78% complete... Commit image migration: 81% complete... Commit image migration: 84% complete... Commit image migration: 87% complete... Commit image migration: 90% complete... Commit image migration: 93% complete... Commit image migration: 96% complete...2023-10-05T21:08:32.363+0000 7f85d89cb700 0 -- 172.21.15.167:0/2512068676 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x7f85bc03aab0 msgr2=0x7f85bc03ced0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:32.372 INFO:tasks.workunit.client.0.smithi167.stderr: Commit image migration: 100% complete...done. 2023-10-05T21:08:32.375 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration prepare test1 2023-10-05T21:08:32.463 INFO:tasks.workunit.client.0.smithi167.stderr:+ expect_fail rbd trash mv test1 2023-10-05T21:08:32.464 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash mv test1 2023-10-05T21:08:32.499 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:32.498+0000 7f7e8f502540 -1 librbd::api::Trash: move: cannot move migrating image to trash 2023-10-05T21:08:32.500 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: deferred delete error: (16) Device or resource busy 2023-10-05T21:08:32.503 INFO:tasks.workunit.client.0.smithi167.stderr:+ return 0 2023-10-05T21:08:32.504 INFO:tasks.workunit.client.0.smithi167.stderr:++ rbd trash ls -a 2023-10-05T21:08:32.504 INFO:tasks.workunit.client.0.smithi167.stderr:++ cut -d ' ' -f 1 2023-10-05T21:08:32.532 INFO:tasks.workunit.client.0.smithi167.stderr:+ ID=19ee8e5ed1af 2023-10-05T21:08:32.532 INFO:tasks.workunit.client.0.smithi167.stderr:+ expect_fail rbd trash rm 19ee8e5ed1af 2023-10-05T21:08:32.532 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash rm 19ee8e5ed1af 2023-10-05T21:08:32.565 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:32.564+0000 7f0171bb9700 -1 librbd::image::RefreshRequest: image being migrated 2023-10-05T21:08:32.565 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:32.564+0000 7f01713b8700 -1 librbd::image::OpenRequest: failed to refresh image: (30) Read-only file system 2023-10-05T21:08:32.565 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:32.564+0000 7f0171bb9700 -1 librbd::ImageState: 0x7f014c04d9c0 failed to open image: (30) Read-only file system 2023-10-05T21:08:32.565 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:32.564+0000 7f0153fff700 -1 librbd::image::RemoveRequest: 0x7f014c000b80 handle_open_image: error opening image: (30) Read-only file system 2023-10-05T21:08:32.567 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: remove error: (30) Read-only file system 2023-10-05T21:08:32.567 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 0% complete...failed. 2023-10-05T21:08:32.569 INFO:tasks.workunit.client.0.smithi167.stderr:+ return 0 2023-10-05T21:08:32.569 INFO:tasks.workunit.client.0.smithi167.stderr:+ expect_fail rbd trash restore 19ee8e5ed1af 2023-10-05T21:08:32.569 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash restore 19ee8e5ed1af 2023-10-05T21:08:32.595 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: restore error: 2023-10-05T21:08:32.594+0000 7f6898c19540 -1 librbd::api::Trash: restore: Current trash source 'migration' does not match expected: user,mirroring,unknown (4) 2023-10-05T21:08:32.596 INFO:tasks.workunit.client.0.smithi167.stderr:(22) Invalid argument 2023-10-05T21:08:32.598 INFO:tasks.workunit.client.0.smithi167.stderr:+ return 0 2023-10-05T21:08:32.598 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration abort test1 2023-10-05T21:08:32.687 INFO:tasks.workunit.client.0.smithi167.stderr: Abort image migration: 1% complete... Abort image migration: 3% complete... Abort image migration: 4% complete... Abort image migration: 6% complete... Abort image migration: 7% complete... Abort image migration: 9% complete... Abort image migration: 10% complete... Abort image migration: 12% complete... Abort image migration: 14% complete... Abort image migration: 15% complete... Abort image migration: 17% complete... Abort image migration: 18% complete... Abort image migration: 20% complete... Abort image migration: 21% complete... Abort image migration: 23% complete... Abort image migration: 25% complete... Abort image migration: 26% complete... Abort image migration: 28% complete... Abort image migration: 29% complete... Abort image migration: 31% complete... Abort image migration: 32% complete... Abort image migration: 34% complete... Abort image migration: 35% complete... Abort image migration: 37% complete... Abort image migration: 39% complete... Abort image migration: 40% complete... Abort image migration: 42% complete... Abort image migration: 43% complete... Abort image migration: 45% complete... Abort image migration: 46% complete... Abort image migration: 48% complete... Abort image migration: 50% complete... Abort image migration: 51% complete... Abort image migration: 53% complete... Abort image migration: 54% complete... Abort image migration: 56% complete... Abort image migration: 57% complete... Abort image migration: 59% complete... Abort image migration: 60% complete... Abort image migration: 62% complete... Abort image migration: 64% complete... Abort image migration: 65% complete... Abort image migration: 67% complete... Abort image migration: 68% complete... Abort image migration: 70% complete... Abort image migration: 71% complete... Abort image migration: 73% complete... Abort image migration: 75% complete... Abort image migration: 76% complete... Abort image migration: 78% complete... Abort image migration: 79% complete... Abort image migration: 81% complete... Abort image migration: 82% complete... Abort image migration: 84% complete... Abort image migration: 85% complete... Abort image migration: 87% complete... Abort image migration: 89% complete... Abort image migration: 90% complete... Abort image migration: 92% complete... Abort image migration: 93% complete... Abort image migration: 95% complete... Abort image migration: 96% complete... Abort image migration: 98% complete...2023-10-05T21:08:32.686+0000 7fa0faa1b700 0 -- 172.21.15.167:0/3511726952 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x55eb35e918c0 msgr2=0x55eb35f838f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:32.707 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:32.706+0000 7fa0faa1b700 0 -- 172.21.15.167:0/3511726952 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7fa0dc03aab0 msgr2=0x7fa0dc03ced0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:32.718 INFO:tasks.workunit.client.0.smithi167.stderr: Abort image migration: 100% complete...done. 2023-10-05T21:08:32.721 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd remove test1 2023-10-05T21:08:32.824 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2023-10-05T21:08:32.827 INFO:tasks.workunit.client.0.smithi167.stderr:+ dd if=/dev/urandom bs=1M count=1 2023-10-05T21:08:32.827 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd --image-format 2 import - test1 2023-10-05T21:08:32.876 INFO:tasks.workunit.client.0.smithi167.stderr:1+0 records in 2023-10-05T21:08:32.877 INFO:tasks.workunit.client.0.smithi167.stderr:1+0 records out 2023-10-05T21:08:32.877 INFO:tasks.workunit.client.0.smithi167.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0483887 s, 21.7 MB/s 2023-10-05T21:08:32.955 INFO:tasks.workunit.client.0.smithi167.stderr: Importing image: 100% complete...done. 2023-10-05T21:08:32.958 INFO:tasks.workunit.client.0.smithi167.stderr:++ rbd export test1 - 2023-10-05T21:08:32.958 INFO:tasks.workunit.client.0.smithi167.stderr:++ md5sum 2023-10-05T21:08:32.993 INFO:tasks.workunit.client.0.smithi167.stderr: Exporting image: 100% complete...done. 2023-10-05T21:08:32.996 INFO:tasks.workunit.client.0.smithi167.stderr:+ md5sum='c35942c3d883781023adcffd3c9833d0 -' 2023-10-05T21:08:32.996 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap create test1@snap1 2023-10-05T21:08:33.699 INFO:tasks.workunit.client.0.smithi167.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-05T21:08:33.711 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap protect test1@snap1 2023-10-05T21:08:33.752 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap create test1@snap2 2023-10-05T21:08:34.699 INFO:tasks.workunit.client.0.smithi167.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-05T21:08:34.710 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd clone test1@snap1 clone_v1 --rbd_default_clone_format=1 2023-10-05T21:08:34.787 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd clone test1@snap2 clone_v2 --rbd_default_clone_format=2 2023-10-05T21:08:34.852 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info clone_v1 2023-10-05T21:08:34.852 INFO:tasks.workunit.client.0.smithi167.stderr:+ fgrep 'parent: rbd/test1@snap1' 2023-10-05T21:08:34.893 INFO:tasks.workunit.client.0.smithi167.stdout: parent: rbd/test1@snap1 2023-10-05T21:08:34.894 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info clone_v2 2023-10-05T21:08:34.894 INFO:tasks.workunit.client.0.smithi167.stderr:+ fgrep 'parent: rbd/test1@snap2' 2023-10-05T21:08:34.933 INFO:tasks.workunit.client.0.smithi167.stdout: parent: rbd/test1@snap2 2023-10-05T21:08:34.934 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info clone_v2 2023-10-05T21:08:34.934 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'op_features: clone-child' 2023-10-05T21:08:34.975 INFO:tasks.workunit.client.0.smithi167.stdout: op_features: clone-child 2023-10-05T21:08:34.975 INFO:tasks.workunit.client.0.smithi167.stderr:++ rbd export clone_v1 - 2023-10-05T21:08:34.976 INFO:tasks.workunit.client.0.smithi167.stderr:++ md5sum 2023-10-05T21:08:35.017 INFO:tasks.workunit.client.0.smithi167.stderr: Exporting image: 100% complete...done. 2023-10-05T21:08:35.020 INFO:tasks.workunit.client.0.smithi167.stderr:+ test 'c35942c3d883781023adcffd3c9833d0 -' = 'c35942c3d883781023adcffd3c9833d0 -' 2023-10-05T21:08:35.021 INFO:tasks.workunit.client.0.smithi167.stderr:++ rbd export clone_v2 - 2023-10-05T21:08:35.021 INFO:tasks.workunit.client.0.smithi167.stderr:++ md5sum 2023-10-05T21:08:35.061 INFO:tasks.workunit.client.0.smithi167.stderr: Exporting image: 100% complete...done. 2023-10-05T21:08:35.065 INFO:tasks.workunit.client.0.smithi167.stderr:+ test 'c35942c3d883781023adcffd3c9833d0 -' = 'c35942c3d883781023adcffd3c9833d0 -' 2023-10-05T21:08:35.065 INFO:tasks.workunit.client.0.smithi167.stderr:++ rbd children test1@snap1 2023-10-05T21:08:35.107 INFO:tasks.workunit.client.0.smithi167.stderr:+ test rbd/clone_v1 = rbd/clone_v1 2023-10-05T21:08:35.107 INFO:tasks.workunit.client.0.smithi167.stderr:++ rbd children test1@snap2 2023-10-05T21:08:35.142 INFO:tasks.workunit.client.0.smithi167.stderr:+ test rbd/clone_v2 = rbd/clone_v2 2023-10-05T21:08:35.142 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration prepare test1 rbd2/test2 2023-10-05T21:08:35.225 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:35.224+0000 7f9d07ca0700 0 -- 172.21.15.167:0/4125570835 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x7f9ce4008740 msgr2=0x7f9ce400abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:36.600 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:36.599+0000 7f9d07ca0700 0 -- 172.21.15.167:0/4125570835 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x7f9cec03a9d0 msgr2=0x7f9cec03cdd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:36.678 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info clone_v1 2023-10-05T21:08:36.678 INFO:tasks.workunit.client.0.smithi167.stderr:+ fgrep 'parent: rbd2/test2@snap1' 2023-10-05T21:08:36.734 INFO:tasks.workunit.client.0.smithi167.stdout: parent: rbd2/test2@snap1 2023-10-05T21:08:36.735 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info clone_v2 2023-10-05T21:08:36.735 INFO:tasks.workunit.client.0.smithi167.stderr:+ fgrep 'parent: rbd2/test2@snap2' 2023-10-05T21:08:36.789 INFO:tasks.workunit.client.0.smithi167.stdout: parent: rbd2/test2@snap2 2023-10-05T21:08:36.789 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info clone_v2 2023-10-05T21:08:36.789 INFO:tasks.workunit.client.0.smithi167.stderr:+ fgrep 'op_features: clone-child' 2023-10-05T21:08:36.842 INFO:tasks.workunit.client.0.smithi167.stdout: op_features: clone-child 2023-10-05T21:08:36.842 INFO:tasks.workunit.client.0.smithi167.stderr:++ rbd children rbd2/test2@snap1 2023-10-05T21:08:36.897 INFO:tasks.workunit.client.0.smithi167.stderr:+ test rbd/clone_v1 = rbd/clone_v1 2023-10-05T21:08:36.897 INFO:tasks.workunit.client.0.smithi167.stderr:++ rbd children rbd2/test2@snap2 2023-10-05T21:08:36.946 INFO:tasks.workunit.client.0.smithi167.stderr:+ test rbd/clone_v2 = rbd/clone_v2 2023-10-05T21:08:36.946 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration execute test1 2023-10-05T21:08:37.045 INFO:tasks.workunit.client.0.smithi167.stderr: Image migration: 100% complete...done. 2023-10-05T21:08:37.180 INFO:tasks.workunit.client.0.smithi167.stderr:+ expect_fail rbd migration commit test1 2023-10-05T21:08:37.180 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration commit test1 2023-10-05T21:08:37.180 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: the image has descendants: 2023-10-05T21:08:37.180 INFO:tasks.workunit.client.0.smithi167.stderr: rbd/clone_v1 2023-10-05T21:08:37.180 INFO:tasks.workunit.client.0.smithi167.stderr: rbd/clone_v2 2023-10-05T21:08:37.180 INFO:tasks.workunit.client.0.smithi167.stderr:Warning: in-use, read-only descendant images will not detect the parent update. 2023-10-05T21:08:37.181 INFO:tasks.workunit.client.0.smithi167.stderr:Ensure no descendant images are opened read-only and run again with force flag. 2023-10-05T21:08:37.181 INFO:tasks.workunit.client.0.smithi167.stderr:+ return 0 2023-10-05T21:08:37.181 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration commit test1 --force 2023-10-05T21:08:37.186 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: the image has descendants: 2023-10-05T21:08:37.186 INFO:tasks.workunit.client.0.smithi167.stderr: rbd/clone_v1 2023-10-05T21:08:37.187 INFO:tasks.workunit.client.0.smithi167.stderr: rbd/clone_v2 2023-10-05T21:08:37.187 INFO:tasks.workunit.client.0.smithi167.stderr:Warning: in-use, read-only descendant images will not detect the parent update. 2023-10-05T21:08:37.187 INFO:tasks.workunit.client.0.smithi167.stderr:Proceeding anyway due to force flag set. 2023-10-05T21:08:37.199 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:37.199+0000 7f7863ffc700 0 -- 172.21.15.167:0/1499067452 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x56421a9a6040 msgr2=0x56421a9a8420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:37.206 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:37.205+0000 7f7863ffc700 0 -- 172.21.15.167:0/1499067452 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x7f784803a900 msgr2=0x7f784803cd00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:38.748 INFO:tasks.workunit.client.0.smithi167.stderr: Commit image migration: 100% complete...done. 2023-10-05T21:08:38.752 INFO:tasks.workunit.client.0.smithi167.stderr:++ rbd export clone_v1 - 2023-10-05T21:08:38.752 INFO:tasks.workunit.client.0.smithi167.stderr:++ md5sum 2023-10-05T21:08:38.797 INFO:tasks.workunit.client.0.smithi167.stderr: Exporting image: 100% complete...done. 2023-10-05T21:08:38.800 INFO:tasks.workunit.client.0.smithi167.stderr:+ test 'c35942c3d883781023adcffd3c9833d0 -' = 'c35942c3d883781023adcffd3c9833d0 -' 2023-10-05T21:08:38.801 INFO:tasks.workunit.client.0.smithi167.stderr:++ rbd export clone_v2 - 2023-10-05T21:08:38.801 INFO:tasks.workunit.client.0.smithi167.stderr:++ md5sum 2023-10-05T21:08:38.843 INFO:tasks.workunit.client.0.smithi167.stderr: Exporting image: 100% complete...done. 2023-10-05T21:08:38.846 INFO:tasks.workunit.client.0.smithi167.stderr:+ test 'c35942c3d883781023adcffd3c9833d0 -' = 'c35942c3d883781023adcffd3c9833d0 -' 2023-10-05T21:08:38.846 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration prepare rbd2/test2 test1 2023-10-05T21:08:38.927 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:38.926+0000 7fe09a641700 0 -- 172.21.15.167:0/793844947 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x7fe080008740 msgr2=0x7fe08000abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:40.717 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:40.716+0000 7fe09a641700 0 -- 172.21.15.167:0/793844947 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x7fe07c03ab30 msgr2=0x7fe07c03cf50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:40.793 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info clone_v1 2023-10-05T21:08:40.793 INFO:tasks.workunit.client.0.smithi167.stderr:+ fgrep 'parent: rbd/test1@snap1' 2023-10-05T21:08:40.849 INFO:tasks.workunit.client.0.smithi167.stdout: parent: rbd/test1@snap1 2023-10-05T21:08:40.849 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info clone_v2 2023-10-05T21:08:40.849 INFO:tasks.workunit.client.0.smithi167.stderr:+ fgrep 'parent: rbd/test1@snap2' 2023-10-05T21:08:40.902 INFO:tasks.workunit.client.0.smithi167.stdout: parent: rbd/test1@snap2 2023-10-05T21:08:40.902 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info clone_v2 2023-10-05T21:08:40.902 INFO:tasks.workunit.client.0.smithi167.stderr:+ fgrep 'op_features: clone-child' 2023-10-05T21:08:40.955 INFO:tasks.workunit.client.0.smithi167.stdout: op_features: clone-child 2023-10-05T21:08:40.956 INFO:tasks.workunit.client.0.smithi167.stderr:++ rbd children test1@snap1 2023-10-05T21:08:41.009 INFO:tasks.workunit.client.0.smithi167.stderr:+ test rbd/clone_v1 = rbd/clone_v1 2023-10-05T21:08:41.009 INFO:tasks.workunit.client.0.smithi167.stderr:++ rbd children test1@snap2 2023-10-05T21:08:41.057 INFO:tasks.workunit.client.0.smithi167.stderr:+ test rbd/clone_v2 = rbd/clone_v2 2023-10-05T21:08:41.058 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration execute test1 2023-10-05T21:08:41.174 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:41.173+0000 7f9b627ed700 0 -- 172.21.15.167:0/4066800316 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x7f9b48008740 msgr2=0x7f9b4800abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:41.175 INFO:tasks.workunit.client.0.smithi167.stderr: Image migration: 100% complete...done. 2023-10-05T21:08:41.177 INFO:tasks.workunit.client.0.smithi167.stderr:+ expect_fail rbd migration commit test1 2023-10-05T21:08:41.177 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration commit test1 2023-10-05T21:08:41.259 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: the image has descendants: 2023-10-05T21:08:41.259 INFO:tasks.workunit.client.0.smithi167.stderr: rbd/clone_v1 2023-10-05T21:08:41.259 INFO:tasks.workunit.client.0.smithi167.stderr: rbd/clone_v2 2023-10-05T21:08:41.260 INFO:tasks.workunit.client.0.smithi167.stderr:Warning: in-use, read-only descendant images will not detect the parent update. 2023-10-05T21:08:41.260 INFO:tasks.workunit.client.0.smithi167.stderr:Ensure no descendant images are opened read-only and run again with force flag. 2023-10-05T21:08:41.262 INFO:tasks.workunit.client.0.smithi167.stderr:+ return 0 2023-10-05T21:08:41.262 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration commit test1 --force 2023-10-05T21:08:41.336 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:41.335+0000 7fa5d3a01700 0 -- 172.21.15.167:0/2484779819 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x564d74946930 msgr2=0x564d749a7fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:41.336 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: the image has descendants: 2023-10-05T21:08:41.336 INFO:tasks.workunit.client.0.smithi167.stderr: rbd/clone_v1 2023-10-05T21:08:41.336 INFO:tasks.workunit.client.0.smithi167.stderr: rbd/clone_v2 2023-10-05T21:08:41.336 INFO:tasks.workunit.client.0.smithi167.stderr:Warning: in-use, read-only descendant images will not detect the parent update. 2023-10-05T21:08:41.336 INFO:tasks.workunit.client.0.smithi167.stderr:Proceeding anyway due to force flag set. 2023-10-05T21:08:41.348 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:41.347+0000 7fa5d3200700 0 -- 172.21.15.167:0/2484779819 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x7fa5b803a960 msgr2=0x7fa5b803cd80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:42.634 INFO:tasks.workunit.client.0.smithi167.stderr: Commit image migration: 100% complete...done. 2023-10-05T21:08:42.637 INFO:tasks.workunit.client.0.smithi167.stderr:++ rbd export clone_v1 - 2023-10-05T21:08:42.638 INFO:tasks.workunit.client.0.smithi167.stderr:++ md5sum 2023-10-05T21:08:42.688 INFO:tasks.workunit.client.0.smithi167.stderr: Exporting image: 100% complete...done. 2023-10-05T21:08:42.691 INFO:tasks.workunit.client.0.smithi167.stderr:+ test 'c35942c3d883781023adcffd3c9833d0 -' = 'c35942c3d883781023adcffd3c9833d0 -' 2023-10-05T21:08:42.692 INFO:tasks.workunit.client.0.smithi167.stderr:++ rbd export clone_v2 - 2023-10-05T21:08:42.692 INFO:tasks.workunit.client.0.smithi167.stderr:++ md5sum 2023-10-05T21:08:42.736 INFO:tasks.workunit.client.0.smithi167.stderr: Exporting image: 100% complete...done. 2023-10-05T21:08:42.740 INFO:tasks.workunit.client.0.smithi167.stderr:+ test 'c35942c3d883781023adcffd3c9833d0 -' = 'c35942c3d883781023adcffd3c9833d0 -' 2023-10-05T21:08:42.740 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd remove clone_v1 2023-10-05T21:08:42.831 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:42.830+0000 7f27e7403700 0 -- 172.21.15.167:0/4185966987 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f27c4008740 msgr2=0x7f27c400abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:42.845 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:42.847 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd remove clone_v2 2023-10-05T21:08:42.933 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:42.932+0000 7f67dccd1700 0 -- 172.21.15.167:0/487043030 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f67b8008740 msgr2=0x7f67b800abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:42.940 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:42.943 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap unprotect test1@snap1 2023-10-05T21:08:42.985 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap purge test1 2023-10-05T21:08:44.732 INFO:tasks.workunit.client.0.smithi167.stderr: Removing all snapshots: 50% complete... Removing all snapshots: 100% complete... Removing all snapshots: 100% complete...done. 2023-10-05T21:08:44.747 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test1 2023-10-05T21:08:44.854 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:44.852+0000 7f5e0fd40700 0 -- 172.21.15.167:0/1239319566 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x55c9aee814d0 msgr2=0x55c9aee939a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:44.860 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:44.863 INFO:tasks.workunit.client.0.smithi167.stderr:+ for format in 1 2 2023-10-05T21:08:44.863 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create -s 128M --image-format 1 test2 2023-10-05T21:08:44.887 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: image format 1 is deprecated 2023-10-05T21:08:44.894 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:44.892+0000 7f41446a2540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:44.904 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration prepare test2 --data-pool rbd2 2023-10-05T21:08:44.974 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-10-05T21:08:45.016 INFO:tasks.workunit.client.0.smithi167.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-10-05T21:08:45.019 INFO:tasks.workunit.client.0.smithi167.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-10-05T21:08:45.019 INFO:tasks.workunit.client.0.smithi167.stdout:elapsed: 0 ops: 1 ops/sec: 333.339 bytes/sec: 333 KiB/s 2023-10-05T21:08:45.024 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration abort test2 2023-10-05T21:08:45.095 INFO:tasks.workunit.client.0.smithi167.stderr: Abort image migration: 1% complete... Abort image migration: 3% complete... Abort image migration: 4% complete... Abort image migration: 6% complete... Abort image migration: 7% complete... Abort image migration: 9% complete... Abort image migration: 10% complete... Abort image migration: 12% complete... Abort image migration: 14% complete... Abort image migration: 15% complete... Abort image migration: 17% complete... Abort image migration: 18% complete... Abort image migration: 20% complete... Abort image migration: 21% complete... Abort image migration: 23% complete... Abort image migration: 25% complete... Abort image migration: 26% complete... Abort image migration: 28% complete... Abort image migration: 29% complete... Abort image migration: 31% complete... Abort image migration: 32% complete... Abort image migration: 34% complete... Abort image migration: 35% complete... Abort image migration: 37% complete... Abort image migration: 39% complete... Abort image migration: 40% complete... Abort image migration: 42% complete... Abort image migration: 43% complete... Abort image migration: 45% complete... Abort image migration: 46% complete... Abort image migration: 48% complete... Abort image migration: 50% complete... Abort image migration: 51% complete... Abort image migration: 53% complete... Abort image migration: 54% complete... Abort image migration: 56% complete... Abort image migration: 57% complete... Abort image migration: 59% complete... Abort image migration: 60% complete... Abort image migration: 62% complete... Abort image migration: 64% complete... Abort image migration: 65% complete... Abort image migration: 67% complete... Abort image migration: 68% complete... Abort image migration: 70% complete... Abort image migration: 71% complete... Abort image migration: 73% complete... Abort image migration: 75% complete... Abort image migration: 76% complete... Abort image migration: 78% complete... Abort image migration: 79% complete... Abort image migration: 81% complete... Abort image migration: 82% complete... Abort image migration: 84% complete... Abort image migration: 85% complete... Abort image migration: 87% complete... Abort image migration: 89% complete... Abort image migration: 90% complete... Abort image migration: 92% complete... Abort image migration: 93% complete... Abort image migration: 95% complete... Abort image migration: 96% complete... Abort image migration: 98% complete...2023-10-05T21:08:45.093+0000 7fe805370700 0 -- 172.21.15.167:0/1388626536 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x562166c2fb20 msgr2=0x562166c31f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-05T21:08:45.107 INFO:tasks.workunit.client.0.smithi167.stderr: Abort image migration: 100% complete...done. 2023-10-05T21:08:45.110 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-10-05T21:08:45.146 INFO:tasks.workunit.client.0.smithi167.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-10-05T21:08:45.149 INFO:tasks.workunit.client.0.smithi167.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-10-05T21:08:45.149 INFO:tasks.workunit.client.0.smithi167.stdout:elapsed: 0 ops: 1 ops/sec: 500.009 bytes/sec: 500 KiB/s 2023-10-05T21:08:45.154 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test2 2023-10-05T21:08:45.203 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2023-10-05T21:08:45.206 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create -s 128M --image-format 1 test2 2023-10-05T21:08:45.229 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: image format 1 is deprecated 2023-10-05T21:08:45.235 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:45.234+0000 7f169aaa0540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:45.243 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration prepare test2 --data-pool rbd2 2023-10-05T21:08:45.315 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-10-05T21:08:45.357 INFO:tasks.workunit.client.0.smithi167.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-10-05T21:08:45.361 INFO:tasks.workunit.client.0.smithi167.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-10-05T21:08:45.361 INFO:tasks.workunit.client.0.smithi167.stdout:elapsed: 0 ops: 1 ops/sec: 250.005 bytes/sec: 250 KiB/s 2023-10-05T21:08:45.370 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration execute test2 2023-10-05T21:08:45.456 INFO:tasks.workunit.client.0.smithi167.stderr: Image migration: 3% complete... Image migration: 6% complete... Image migration: 9% complete... Image migration: 12% complete... Image migration: 15% complete... Image migration: 18% complete... Image migration: 21% complete... Image migration: 25% complete... Image migration: 28% complete... Image migration: 31% complete... Image migration: 34% complete... Image migration: 37% complete... Image migration: 40% complete... Image migration: 43% complete... Image migration: 46% complete... Image migration: 50% complete... Image migration: 53% complete... Image migration: 56% complete... Image migration: 59% complete... Image migration: 62% complete... Image migration: 65% complete... Image migration: 68% complete... Image migration: 71% complete... Image migration: 75% complete... Image migration: 78% complete... Image migration: 81% complete... Image migration: 84% complete... Image migration: 87% complete... Image migration: 90% complete... Image migration: 93% complete... Image migration: 96% complete...2023-10-05T21:08:45.453+0000 7f10c78af700 0 -- 172.21.15.167:0/2684454897 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f10a8008b50 msgr2=0x7f10a806bd00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:45.477 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:45.475+0000 7f10c80b0700 0 -- 172.21.15.167:0/2684454897 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x55d16000f5b0 msgr2=0x55d16006d870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:45.485 INFO:tasks.workunit.client.0.smithi167.stderr: Image migration: 100% complete...done. 2023-10-05T21:08:45.488 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration abort test2 2023-10-05T21:08:45.569 INFO:tasks.workunit.client.0.smithi167.stderr: Abort image migration: 1% complete... Abort image migration: 3% complete... Abort image migration: 4% complete... Abort image migration: 6% complete... Abort image migration: 7% complete... Abort image migration: 9% complete... Abort image migration: 10% complete... Abort image migration: 12% complete... Abort image migration: 14% complete... Abort image migration: 15% complete... Abort image migration: 17% complete... Abort image migration: 18% complete... Abort image migration: 20% complete... Abort image migration: 21% complete... Abort image migration: 23% complete... Abort image migration: 25% complete... Abort image migration: 26% complete... Abort image migration: 28% complete... Abort image migration: 29% complete... Abort image migration: 31% complete... Abort image migration: 32% complete... Abort image migration: 34% complete... Abort image migration: 35% complete... Abort image migration: 37% complete... Abort image migration: 39% complete... Abort image migration: 40% complete... Abort image migration: 42% complete... Abort image migration: 43% complete... Abort image migration: 45% complete... Abort image migration: 46% complete... Abort image migration: 48% complete... Abort image migration: 50% complete... Abort image migration: 51% complete... Abort image migration: 53% complete... Abort image migration: 54% complete... Abort image migration: 56% complete... Abort image migration: 57% complete... Abort image migration: 59% complete... Abort image migration: 60% complete... Abort image migration: 62% complete... Abort image migration: 64% complete... Abort image migration: 65% complete... Abort image migration: 67% complete... Abort image migration: 68% complete... Abort image migration: 70% complete... Abort image migration: 71% complete... Abort image migration: 73% complete... Abort image migration: 75% complete... Abort image migration: 76% complete... Abort image migration: 78% complete... Abort image migration: 79% complete... Abort image migration: 81% complete... Abort image migration: 82% complete... Abort image migration: 84% complete... Abort image migration: 85% complete... Abort image migration: 87% complete... Abort image migration: 89% complete... Abort image migration: 90% complete... Abort image migration: 92% complete... Abort image migration: 93% complete... Abort image migration: 95% complete... Abort image migration: 96% complete... Abort image migration: 98% complete...2023-10-05T21:08:45.567+0000 7f44a6632700 0 -- 172.21.15.167:0/1913614479 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x55dc09646580 msgr2=0x55dc096499a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:45.584 INFO:tasks.workunit.client.0.smithi167.stderr: Abort image migration: 100% complete...done. 2023-10-05T21:08:45.586 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-10-05T21:08:45.623 INFO:tasks.workunit.client.0.smithi167.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-10-05T21:08:45.626 INFO:tasks.workunit.client.0.smithi167.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-10-05T21:08:45.626 INFO:tasks.workunit.client.0.smithi167.stdout:elapsed: 0 ops: 1 ops/sec: 333.339 bytes/sec: 333 KiB/s 2023-10-05T21:08:45.630 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test2 2023-10-05T21:08:45.686 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2023-10-05T21:08:45.690 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create -s 128M --image-format 1 test2 2023-10-05T21:08:45.712 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: image format 1 is deprecated 2023-10-05T21:08:45.718 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:45.716+0000 7efe1b65b540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:45.725 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration prepare test2 --data-pool INVALID_DATA_POOL 2023-10-05T21:08:45.770 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:45.768+0000 7fc09fb14540 -1 librbd::image::CreateRequest: 0x5618118e4fd0 validate_data_pool: data pool INVALID_DATA_POOL does not exist 2023-10-05T21:08:45.770 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:45.768+0000 7fc09fb14540 -1 librbd::Migration: create_dst_image: header creation failed: (2) No such file or directory 2023-10-05T21:08:45.776 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: preparing migration failed: (2) No such file or directory 2023-10-05T21:08:45.779 INFO:tasks.workunit.client.0.smithi167.stderr:+ true 2023-10-05T21:08:45.779 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-10-05T21:08:45.816 INFO:tasks.workunit.client.0.smithi167.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-10-05T21:08:45.818 INFO:tasks.workunit.client.0.smithi167.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-10-05T21:08:45.818 INFO:tasks.workunit.client.0.smithi167.stdout:elapsed: 0 ops: 1 ops/sec: 500.009 bytes/sec: 500 KiB/s 2023-10-05T21:08:45.822 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test2 2023-10-05T21:08:45.871 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2023-10-05T21:08:45.874 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create -s 128M --image-format 1 test2 2023-10-05T21:08:45.897 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: image format 1 is deprecated 2023-10-05T21:08:45.903 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:45.901+0000 7f667254e540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:45.911 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration prepare test2 rbd2/test2 2023-10-05T21:08:45.980 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 rbd2/test2 2023-10-05T21:08:46.022 INFO:tasks.workunit.client.0.smithi167.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-10-05T21:08:46.025 INFO:tasks.workunit.client.0.smithi167.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-10-05T21:08:46.025 INFO:tasks.workunit.client.0.smithi167.stdout:elapsed: 0 ops: 1 ops/sec: 500.009 bytes/sec: 500 KiB/s 2023-10-05T21:08:46.029 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration abort test2 2023-10-05T21:08:46.095 INFO:tasks.workunit.client.0.smithi167.stderr: Abort image migration: 1% complete... Abort image migration: 3% complete... Abort image migration: 4% complete... Abort image migration: 6% complete... Abort image migration: 7% complete... Abort image migration: 9% complete... Abort image migration: 10% complete... Abort image migration: 12% complete... Abort image migration: 14% complete... Abort image migration: 15% complete... Abort image migration: 17% complete... Abort image migration: 18% complete... Abort image migration: 20% complete... Abort image migration: 21% complete... Abort image migration: 23% complete... Abort image migration: 25% complete... Abort image migration: 26% complete... Abort image migration: 28% complete... Abort image migration: 29% complete... Abort image migration: 31% complete... Abort image migration: 32% complete... Abort image migration: 34% complete... Abort image migration: 35% complete... Abort image migration: 37% complete... Abort image migration: 39% complete... Abort image migration: 40% complete... Abort image migration: 42% complete... Abort image migration: 43% complete... Abort image migration: 45% complete... Abort image migration: 46% complete... Abort image migration: 48% complete... Abort image migration: 50% complete... Abort image migration: 51% complete... Abort image migration: 53% complete... Abort image migration: 54% complete... Abort image migration: 56% complete... Abort image migration: 57% complete... Abort image migration: 59% complete... Abort image migration: 60% complete... Abort image migration: 62% complete... Abort image migration: 64% complete... Abort image migration: 65% complete... Abort image migration: 67% complete... Abort image migration: 68% complete... Abort image migration: 70% complete... Abort image migration: 71% complete... Abort image migration: 73% complete... Abort image migration: 75% complete... Abort image migration: 76% complete... Abort image migration: 78% complete... Abort image migration: 79% complete... Abort image migration: 81% complete... Abort image migration: 82% complete... Abort image migration: 84% complete... Abort image migration: 85% complete... Abort image migration: 87% complete... Abort image migration: 89% complete... Abort image migration: 90% complete... Abort image migration: 92% complete... Abort image migration: 93% complete... Abort image migration: 95% complete... Abort image migration: 96% complete... Abort image migration: 98% complete... Abort image migration: 100% complete...done. 2023-10-05T21:08:46.097 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-10-05T21:08:46.134 INFO:tasks.workunit.client.0.smithi167.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-10-05T21:08:46.136 INFO:tasks.workunit.client.0.smithi167.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-10-05T21:08:46.136 INFO:tasks.workunit.client.0.smithi167.stdout:elapsed: 0 ops: 1 ops/sec: 500.009 bytes/sec: 500 KiB/s 2023-10-05T21:08:46.140 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test2 2023-10-05T21:08:46.191 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2023-10-05T21:08:46.194 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create -s 128M --image-format 1 test2 2023-10-05T21:08:46.217 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: image format 1 is deprecated 2023-10-05T21:08:46.223 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:46.221+0000 7f187bdf8540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:46.230 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration prepare test2 rbd2/test2 2023-10-05T21:08:46.294 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration abort rbd2/test2 2023-10-05T21:08:46.356 INFO:tasks.workunit.client.0.smithi167.stderr: Abort image migration: 1% complete... Abort image migration: 3% complete... Abort image migration: 4% complete... Abort image migration: 6% complete... Abort image migration: 7% complete... Abort image migration: 9% complete... Abort image migration: 10% complete... Abort image migration: 12% complete... Abort image migration: 14% complete... Abort image migration: 15% complete... Abort image migration: 17% complete... Abort image migration: 18% complete... Abort image migration: 20% complete... Abort image migration: 21% complete... Abort image migration: 23% complete... Abort image migration: 25% complete... Abort image migration: 26% complete... Abort image migration: 28% complete... Abort image migration: 29% complete... Abort image migration: 31% complete... Abort image migration: 32% complete... Abort image migration: 34% complete... Abort image migration: 35% complete... Abort image migration: 37% complete... Abort image migration: 39% complete... Abort image migration: 40% complete... Abort image migration: 42% complete... Abort image migration: 43% complete... Abort image migration: 45% complete... Abort image migration: 46% complete... Abort image migration: 48% complete... Abort image migration: 50% complete... Abort image migration: 51% complete... Abort image migration: 53% complete... Abort image migration: 54% complete... Abort image migration: 56% complete... Abort image migration: 57% complete... Abort image migration: 59% complete... Abort image migration: 60% complete... Abort image migration: 62% complete... Abort image migration: 64% complete... Abort image migration: 65% complete... Abort image migration: 67% complete... Abort image migration: 68% complete... Abort image migration: 70% complete... Abort image migration: 71% complete... Abort image migration: 73% complete... Abort image migration: 75% complete... Abort image migration: 76% complete... Abort image migration: 78% complete... Abort image migration: 79% complete... Abort image migration: 81% complete... Abort image migration: 82% complete... Abort image migration: 84% complete... Abort image migration: 85% complete... Abort image migration: 87% complete... Abort image migration: 89% complete... Abort image migration: 90% complete... Abort image migration: 92% complete... Abort image migration: 93% complete... Abort image migration: 95% complete... Abort image migration: 96% complete... Abort image migration: 98% complete...2023-10-05T21:08:46.354+0000 7f11e7cfe700 0 -- 172.21.15.167:0/2946309498 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x564d96ccd8c0 msgr2=0x564d96dbe7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-05T21:08:46.373 INFO:tasks.workunit.client.0.smithi167.stderr: Abort image migration: 100% complete...done. 2023-10-05T21:08:46.375 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-10-05T21:08:46.412 INFO:tasks.workunit.client.0.smithi167.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-10-05T21:08:46.415 INFO:tasks.workunit.client.0.smithi167.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-10-05T21:08:46.415 INFO:tasks.workunit.client.0.smithi167.stdout:elapsed: 0 ops: 1 ops/sec: 500.009 bytes/sec: 500 KiB/s 2023-10-05T21:08:46.419 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test2 2023-10-05T21:08:46.470 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2023-10-05T21:08:46.473 INFO:tasks.workunit.client.0.smithi167.stderr:+ test 1 = 1 2023-10-05T21:08:46.473 INFO:tasks.workunit.client.0.smithi167.stderr:+ continue 2023-10-05T21:08:46.473 INFO:tasks.workunit.client.0.smithi167.stderr:+ for format in 1 2 2023-10-05T21:08:46.473 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create -s 128M --image-format 2 test2 2023-10-05T21:08:46.516 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration prepare test2 --data-pool rbd2 2023-10-05T21:08:46.632 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:46.630+0000 7f1935c0f700 0 -- 172.21.15.167:0/3041087369 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x564d7892dd50 msgr2=0x564d78a1c970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:46.638 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-10-05T21:08:46.688 INFO:tasks.workunit.client.0.smithi167.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-10-05T21:08:46.693 INFO:tasks.workunit.client.0.smithi167.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-10-05T21:08:46.693 INFO:tasks.workunit.client.0.smithi167.stdout:elapsed: 0 ops: 1 ops/sec: 200.004 bytes/sec: 200 KiB/s 2023-10-05T21:08:46.701 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration abort test2 2023-10-05T21:08:47.002 INFO:tasks.workunit.client.0.smithi167.stderr: Abort image migration: 1% complete... Abort image migration: 3% complete... Abort image migration: 4% complete... Abort image migration: 6% complete... Abort image migration: 7% complete... Abort image migration: 9% complete... Abort image migration: 10% complete... Abort image migration: 12% complete... Abort image migration: 14% complete... Abort image migration: 15% complete... Abort image migration: 17% complete... Abort image migration: 18% complete... Abort image migration: 20% complete... Abort image migration: 21% complete... Abort image migration: 23% complete... Abort image migration: 25% complete... Abort image migration: 26% complete... Abort image migration: 28% complete... Abort image migration: 29% complete... Abort image migration: 31% complete... Abort image migration: 32% complete... Abort image migration: 34% complete... Abort image migration: 35% complete... Abort image migration: 37% complete... Abort image migration: 39% complete... Abort image migration: 40% complete... Abort image migration: 42% complete... Abort image migration: 43% complete... Abort image migration: 45% complete... Abort image migration: 46% complete... Abort image migration: 48% complete... Abort image migration: 50% complete... Abort image migration: 51% complete... Abort image migration: 53% complete... Abort image migration: 54% complete... Abort image migration: 56% complete... Abort image migration: 57% complete... Abort image migration: 59% complete... Abort image migration: 60% complete... Abort image migration: 62% complete... Abort image migration: 64% complete... Abort image migration: 65% complete... Abort image migration: 67% complete... Abort image migration: 68% complete... Abort image migration: 70% complete... Abort image migration: 71% complete... Abort image migration: 73% complete... Abort image migration: 75% complete... Abort image migration: 76% complete... Abort image migration: 78% complete... Abort image migration: 79% complete... Abort image migration: 81% complete... Abort image migration: 82% complete... Abort image migration: 84% complete... Abort image migration: 85% complete... Abort image migration: 87% complete... Abort image migration: 89% complete... Abort image migration: 90% complete... Abort image migration: 92% complete... Abort image migration: 93% complete... Abort image migration: 95% complete... Abort image migration: 96% complete... Abort image migration: 98% complete...2023-10-05T21:08:47.000+0000 7f57e7b23700 0 -- 172.21.15.167:0/2729189209 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f57c40064d0 msgr2=0x7f57c4005eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:47.010 INFO:tasks.workunit.client.0.smithi167.stderr: Abort image migration: 100% complete...done. 2023-10-05T21:08:47.013 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-10-05T21:08:47.056 INFO:tasks.workunit.client.0.smithi167.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-10-05T21:08:47.057 INFO:tasks.workunit.client.0.smithi167.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-10-05T21:08:47.057 INFO:tasks.workunit.client.0.smithi167.stdout:elapsed: 0 ops: 1 ops/sec: 1000.02 bytes/sec: 1000 KiB/s 2023-10-05T21:08:47.065 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test2 2023-10-05T21:08:47.176 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete...2023-10-05T21:08:47.174+0000 7f0643c3c700 0 -- 172.21.15.167:0/3450627092 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x5588335dd4d0 msgr2=0x5588335eeea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-05T21:08:47.185 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:47.188 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create -s 128M --image-format 2 test2 2023-10-05T21:08:47.231 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration prepare test2 --data-pool rbd2 2023-10-05T21:08:47.333 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:47.331+0000 7fe1ae43e700 0 -- 172.21.15.167:0/703139440 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x562bb3afdd50 msgr2=0x562bb3becb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:47.338 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-10-05T21:08:47.389 INFO:tasks.workunit.client.0.smithi167.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-10-05T21:08:47.402 INFO:tasks.workunit.client.0.smithi167.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-10-05T21:08:47.403 INFO:tasks.workunit.client.0.smithi167.stdout:elapsed: 0 ops: 1 ops/sec: 71.4299 bytes/sec: 71 KiB/s 2023-10-05T21:08:47.419 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration execute test2 2023-10-05T21:08:47.521 INFO:tasks.workunit.client.0.smithi167.stderr: Image migration: 3% complete... Image migration: 6% complete... Image migration: 9% complete... Image migration: 12% complete... Image migration: 15% complete... Image migration: 18% complete... Image migration: 21% complete... Image migration: 25% complete... Image migration: 28% complete... Image migration: 31% complete... Image migration: 34% complete... Image migration: 37% complete... Image migration: 40% complete... Image migration: 43% complete... Image migration: 46% complete... Image migration: 50% complete... Image migration: 53% complete... Image migration: 56% complete... Image migration: 59% complete... Image migration: 62% complete... Image migration: 65% complete... Image migration: 68% complete... Image migration: 71% complete... Image migration: 75% complete... Image migration: 78% complete... Image migration: 81% complete... Image migration: 84% complete... Image migration: 87% complete... Image migration: 90% complete... Image migration: 93% complete... Image migration: 96% complete...2023-10-05T21:08:47.519+0000 7f17cd985700 0 -- 172.21.15.167:0/2124573007 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f17a8005830 msgr2=0x7f17a8005ca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:47.546 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:47.544+0000 7f17ce186700 0 -- 172.21.15.167:0/2124573007 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x55f19ce00440 msgr2=0x55f19cef69b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:47.551 INFO:tasks.workunit.client.0.smithi167.stderr: Image migration: 100% complete...done. 2023-10-05T21:08:47.554 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration abort test2 2023-10-05T21:08:47.669 INFO:tasks.workunit.client.0.smithi167.stderr: Abort image migration: 1% complete... Abort image migration: 3% complete... Abort image migration: 4% complete... Abort image migration: 6% complete... Abort image migration: 7% complete... Abort image migration: 9% complete... Abort image migration: 10% complete... Abort image migration: 12% complete... Abort image migration: 14% complete... Abort image migration: 15% complete... Abort image migration: 17% complete... Abort image migration: 18% complete... Abort image migration: 20% complete... Abort image migration: 21% complete... Abort image migration: 23% complete... Abort image migration: 25% complete... Abort image migration: 26% complete... Abort image migration: 28% complete... Abort image migration: 29% complete... Abort image migration: 31% complete... Abort image migration: 32% complete... Abort image migration: 34% complete... Abort image migration: 35% complete... Abort image migration: 37% complete... Abort image migration: 39% complete... Abort image migration: 40% complete... Abort image migration: 42% complete... Abort image migration: 43% complete... Abort image migration: 45% complete... Abort image migration: 46% complete... Abort image migration: 48% complete... Abort image migration: 50% complete... Abort image migration: 51% complete... Abort image migration: 53% complete... Abort image migration: 54% complete... Abort image migration: 56% complete... Abort image migration: 57% complete... Abort image migration: 59% complete... Abort image migration: 60% complete... Abort image migration: 62% complete... Abort image migration: 64% complete... Abort image migration: 65% complete... Abort image migration: 67% complete... Abort image migration: 68% complete... Abort image migration: 70% complete... Abort image migration: 71% complete... Abort image migration: 73% complete... Abort image migration: 75% complete... Abort image migration: 76% complete... Abort image migration: 78% complete... Abort image migration: 79% complete... Abort image migration: 81% complete... Abort image migration: 82% complete... Abort image migration: 84% complete... Abort image migration: 85% complete... Abort image migration: 87% complete... Abort image migration: 89% complete... Abort image migration: 90% complete... Abort image migration: 92% complete... Abort image migration: 93% complete... Abort image migration: 95% complete... Abort image migration: 96% complete... Abort image migration: 98% complete...2023-10-05T21:08:47.667+0000 7fcaf6075700 0 -- 172.21.15.167:0/2158025396 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7fcad0006340 msgr2=0x7fcad0008720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:47.671 INFO:tasks.workunit.client.0.smithi167.stderr: Abort image migration: 100% complete...done. 2023-10-05T21:08:47.674 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-10-05T21:08:47.716 INFO:tasks.workunit.client.0.smithi167.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-10-05T21:08:47.717 INFO:tasks.workunit.client.0.smithi167.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-10-05T21:08:47.717 INFO:tasks.workunit.client.0.smithi167.stdout:elapsed: 0 ops: 1 ops/sec: 500.009 bytes/sec: 500 KiB/s 2023-10-05T21:08:47.725 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test2 2023-10-05T21:08:47.831 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete...2023-10-05T21:08:47.829+0000 7fa05d096700 0 -- 172.21.15.167:0/3125992251 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x555812780870 msgr2=0x5558128248f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:47.836 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:47.839 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create -s 128M --image-format 2 test2 2023-10-05T21:08:47.882 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration prepare test2 --data-pool INVALID_DATA_POOL 2023-10-05T21:08:47.950 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:47.948+0000 7f5dc876f540 -1 librbd::image::CreateRequest: 0x564f908076f0 validate_data_pool: data pool INVALID_DATA_POOL does not exist 2023-10-05T21:08:47.950 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:47.948+0000 7f5dc876f540 -1 librbd::Migration: create_dst_image: header creation failed: (2) No such file or directory 2023-10-05T21:08:47.970 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: preparing migration failed: (2) No such file or directory 2023-10-05T21:08:47.973 INFO:tasks.workunit.client.0.smithi167.stderr:+ true 2023-10-05T21:08:47.973 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-10-05T21:08:48.015 INFO:tasks.workunit.client.0.smithi167.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-10-05T21:08:48.018 INFO:tasks.workunit.client.0.smithi167.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-10-05T21:08:48.019 INFO:tasks.workunit.client.0.smithi167.stdout:elapsed: 0 ops: 1 ops/sec: 250.005 bytes/sec: 250 KiB/s 2023-10-05T21:08:48.026 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test2 2023-10-05T21:08:48.134 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete...2023-10-05T21:08:48.132+0000 7f8d808cc700 0 -- 172.21.15.167:0/4116747464 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x562c1aab3870 msgr2=0x562c1ab589f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:48.138 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:48.141 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create -s 128M --image-format 2 test2 2023-10-05T21:08:48.183 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration prepare test2 rbd2/test2 2023-10-05T21:08:48.275 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:48.273+0000 7fa8966b7700 0 -- 172.21.15.167:0/3791595678 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x55ed3e18ded0 msgr2=0x55ed3e27b890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:48.279 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 rbd2/test2 2023-10-05T21:08:48.328 INFO:tasks.workunit.client.0.smithi167.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-10-05T21:08:48.331 INFO:tasks.workunit.client.0.smithi167.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-10-05T21:08:48.331 INFO:tasks.workunit.client.0.smithi167.stdout:elapsed: 0 ops: 1 ops/sec: 333.339 bytes/sec: 333 KiB/s 2023-10-05T21:08:48.337 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration abort test2 2023-10-05T21:08:48.448 INFO:tasks.workunit.client.0.smithi167.stderr: Abort image migration: 1% complete... Abort image migration: 3% complete... Abort image migration: 4% complete... Abort image migration: 6% complete... Abort image migration: 7% complete... Abort image migration: 9% complete... Abort image migration: 10% complete... Abort image migration: 12% complete... Abort image migration: 14% complete... Abort image migration: 15% complete... Abort image migration: 17% complete... Abort image migration: 18% complete... Abort image migration: 20% complete... Abort image migration: 21% complete... Abort image migration: 23% complete... Abort image migration: 25% complete... Abort image migration: 26% complete... Abort image migration: 28% complete... Abort image migration: 29% complete... Abort image migration: 31% complete... Abort image migration: 32% complete... Abort image migration: 34% complete... Abort image migration: 35% complete... Abort image migration: 37% complete... Abort image migration: 39% complete... Abort image migration: 40% complete... Abort image migration: 42% complete... Abort image migration: 43% complete... Abort image migration: 45% complete... Abort image migration: 46% complete... Abort image migration: 48% complete... Abort image migration: 50% complete... Abort image migration: 51% complete... Abort image migration: 53% complete... Abort image migration: 54% complete... Abort image migration: 56% complete... Abort image migration: 57% complete... Abort image migration: 59% complete... Abort image migration: 60% complete... Abort image migration: 62% complete... Abort image migration: 64% complete... Abort image migration: 65% complete... Abort image migration: 67% complete... Abort image migration: 68% complete... Abort image migration: 70% complete... Abort image migration: 71% complete... Abort image migration: 73% complete... Abort image migration: 75% complete... Abort image migration: 76% complete... Abort image migration: 78% complete... Abort image migration: 79% complete... Abort image migration: 81% complete... Abort image migration: 82% complete... Abort image migration: 84% complete... Abort image migration: 85% complete... Abort image migration: 87% complete... Abort image migration: 89% complete... Abort image migration: 90% complete... Abort image migration: 92% complete... Abort image migration: 93% complete... Abort image migration: 95% complete... Abort image migration: 96% complete... Abort image migration: 98% complete... Abort image migration: 100% complete...done. 2023-10-05T21:08:48.451 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-10-05T21:08:48.495 INFO:tasks.workunit.client.0.smithi167.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-10-05T21:08:48.497 INFO:tasks.workunit.client.0.smithi167.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-10-05T21:08:48.497 INFO:tasks.workunit.client.0.smithi167.stdout:elapsed: 0 ops: 1 ops/sec: 500.009 bytes/sec: 500 KiB/s 2023-10-05T21:08:48.504 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test2 2023-10-05T21:08:48.598 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete...2023-10-05T21:08:48.596+0000 7f446668b700 0 -- 172.21.15.167:0/1980113932 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x557d907388d0 msgr2=0x557d907dcec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-05T21:08:48.607 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:48.610 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create -s 128M --image-format 2 test2 2023-10-05T21:08:48.659 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration prepare test2 rbd2/test2 2023-10-05T21:08:48.777 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:48.775+0000 7fa666cd9700 0 -- 172.21.15.167:0/2762820535 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7fa644008740 msgr2=0x7fa64400abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:48.786 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration abort rbd2/test2 2023-10-05T21:08:48.867 INFO:tasks.workunit.client.0.smithi167.stderr: Abort image migration: 1% complete... Abort image migration: 3% complete... Abort image migration: 4% complete... Abort image migration: 6% complete... Abort image migration: 7% complete... Abort image migration: 9% complete... Abort image migration: 10% complete... Abort image migration: 12% complete... Abort image migration: 14% complete... Abort image migration: 15% complete... Abort image migration: 17% complete... Abort image migration: 18% complete... Abort image migration: 20% complete... Abort image migration: 21% complete... Abort image migration: 23% complete... Abort image migration: 25% complete... Abort image migration: 26% complete... Abort image migration: 28% complete... Abort image migration: 29% complete... Abort image migration: 31% complete... Abort image migration: 32% complete... Abort image migration: 34% complete... Abort image migration: 35% complete... Abort image migration: 37% complete... Abort image migration: 39% complete... Abort image migration: 40% complete... Abort image migration: 42% complete... Abort image migration: 43% complete... Abort image migration: 45% complete... Abort image migration: 46% complete... Abort image migration: 48% complete... Abort image migration: 50% complete... Abort image migration: 51% complete... Abort image migration: 53% complete... Abort image migration: 54% complete... Abort image migration: 56% complete... Abort image migration: 57% complete... Abort image migration: 59% complete... Abort image migration: 60% complete... Abort image migration: 62% complete... Abort image migration: 64% complete... Abort image migration: 65% complete... Abort image migration: 67% complete... Abort image migration: 68% complete... Abort image migration: 70% complete... Abort image migration: 71% complete... Abort image migration: 73% complete... Abort image migration: 75% complete... Abort image migration: 76% complete... Abort image migration: 78% complete... Abort image migration: 79% complete... Abort image migration: 81% complete... Abort image migration: 82% complete... Abort image migration: 84% complete... Abort image migration: 85% complete... Abort image migration: 87% complete... Abort image migration: 89% complete... Abort image migration: 90% complete... Abort image migration: 92% complete... Abort image migration: 93% complete... Abort image migration: 95% complete... Abort image migration: 96% complete... Abort image migration: 98% complete... Abort image migration: 100% complete...done. 2023-10-05T21:08:48.870 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-10-05T21:08:48.911 INFO:tasks.workunit.client.0.smithi167.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-10-05T21:08:48.914 INFO:tasks.workunit.client.0.smithi167.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-10-05T21:08:48.914 INFO:tasks.workunit.client.0.smithi167.stdout:elapsed: 0 ops: 1 ops/sec: 333.339 bytes/sec: 333 KiB/s 2023-10-05T21:08:48.922 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test2 2023-10-05T21:08:49.018 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete...2023-10-05T21:08:49.016+0000 7fe2dcc23700 0 -- 172.21.15.167:0/1985012561 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7fe2b8008740 msgr2=0x7fe2b800abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-05T21:08:49.026 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:49.029 INFO:tasks.workunit.client.0.smithi167.stderr:+ test 2 = 1 2023-10-05T21:08:49.029 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create -s 128M --image-format 2 test2 2023-10-05T21:08:49.072 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration prepare test2 rbd2/ns1/test3 2023-10-05T21:08:49.177 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:49.175+0000 7fe773edd700 0 -- 172.21.15.167:0/290089058 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x556c80ac8ed0 msgr2=0x556c80b6a6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:49.182 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 rbd2/ns1/test3 2023-10-05T21:08:49.230 INFO:tasks.workunit.client.0.smithi167.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-10-05T21:08:49.233 INFO:tasks.workunit.client.0.smithi167.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-10-05T21:08:49.233 INFO:tasks.workunit.client.0.smithi167.stdout:elapsed: 0 ops: 1 ops/sec: 333.339 bytes/sec: 333 KiB/s 2023-10-05T21:08:49.239 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd migration abort test2 2023-10-05T21:08:49.325 INFO:tasks.workunit.client.0.smithi167.stderr: Abort image migration: 1% complete... Abort image migration: 3% complete... Abort image migration: 4% complete... Abort image migration: 6% complete... Abort image migration: 7% complete... Abort image migration: 9% complete... Abort image migration: 10% complete... Abort image migration: 12% complete... Abort image migration: 14% complete... Abort image migration: 15% complete... Abort image migration: 17% complete... Abort image migration: 18% complete... Abort image migration: 20% complete... Abort image migration: 21% complete... Abort image migration: 23% complete... Abort image migration: 25% complete... Abort image migration: 26% complete... Abort image migration: 28% complete... Abort image migration: 29% complete... Abort image migration: 31% complete... Abort image migration: 32% complete... Abort image migration: 34% complete... Abort image migration: 35% complete... Abort image migration: 37% complete... Abort image migration: 39% complete... Abort image migration: 40% complete... Abort image migration: 42% complete... Abort image migration: 43% complete... Abort image migration: 45% complete... Abort image migration: 46% complete... Abort image migration: 48% complete... Abort image migration: 50% complete... Abort image migration: 51% complete... Abort image migration: 53% complete... Abort image migration: 54% complete... Abort image migration: 56% complete... Abort image migration: 57% complete... Abort image migration: 59% complete... Abort image migration: 60% complete... Abort image migration: 62% complete... Abort image migration: 64% complete... Abort image migration: 65% complete... Abort image migration: 67% complete... Abort image migration: 68% complete... Abort image migration: 70% complete... Abort image migration: 71% complete... Abort image migration: 73% complete... Abort image migration: 75% complete... Abort image migration: 76% complete... Abort image migration: 78% complete... Abort image migration: 79% complete... Abort image migration: 81% complete... Abort image migration: 82% complete... Abort image migration: 84% complete... Abort image migration: 85% complete... Abort image migration: 87% complete... Abort image migration: 89% complete... Abort image migration: 90% complete... Abort image migration: 92% complete... Abort image migration: 93% complete... Abort image migration: 95% complete... Abort image migration: 96% complete... Abort image migration: 98% complete... Abort image migration: 100% complete...done. 2023-10-05T21:08:49.327 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd bench --io-type write --io-size 1024 --io-total 1024 test2 2023-10-05T21:08:49.369 INFO:tasks.workunit.client.0.smithi167.stdout:bench type write io_size 1024 io_threads 16 bytes 1024 pattern sequential 2023-10-05T21:08:49.371 INFO:tasks.workunit.client.0.smithi167.stdout: SEC OPS OPS/SEC BYTES/SEC 2023-10-05T21:08:49.371 INFO:tasks.workunit.client.0.smithi167.stdout:elapsed: 0 ops: 1 ops/sec: 500.009 bytes/sec: 500 KiB/s 2023-10-05T21:08:49.379 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test2 2023-10-05T21:08:49.483 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete...2023-10-05T21:08:49.482+0000 7f2ecbd62700 0 -- 172.21.15.167:0/1332544879 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f2ea8008740 msgr2=0x7f2ea800abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:49.494 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:49.497 INFO:tasks.workunit.client.0.smithi167.stderr:+ remove_images 2023-10-05T21:08:49.497 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:49.578 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:49.655 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:49.738 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:49.819 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:49.896 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:49.977 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:50.058 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:50.136 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:50.214 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:50.294 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:50.373 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:50.452 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:50.535 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:50.617 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:50.896 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:50.977 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:51.054 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:51.136 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph osd pool rm rbd2 rbd2 --yes-i-really-really-mean-it 2023-10-05T21:08:51.718 INFO:tasks.workunit.client.0.smithi167.stderr:pool 'rbd2' does not exist 2023-10-05T21:08:51.728 INFO:tasks.workunit.client.0.smithi167.stderr:+ test_config 2023-10-05T21:08:51.728 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'testing config...' 2023-10-05T21:08:51.728 INFO:tasks.workunit.client.0.smithi167.stdout:testing config... 2023-10-05T21:08:51.728 INFO:tasks.workunit.client.0.smithi167.stderr:+ remove_images 2023-10-05T21:08:51.729 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:51.812 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:51.890 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:51.969 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:52.047 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:52.133 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:52.213 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:52.290 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:52.368 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:52.443 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:52.519 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:52.595 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:52.676 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:52.755 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:52.833 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:52.908 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:52.984 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:53.060 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:53.137 INFO:tasks.workunit.client.0.smithi167.stderr:+ expect_fail rbd config global set osd rbd_cache true 2023-10-05T21:08:53.137 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd config global set osd rbd_cache true 2023-10-05T21:08:53.160 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: invalid config entity: osd (must be global, client or client.) 2023-10-05T21:08:53.161 INFO:tasks.workunit.client.0.smithi167.stderr:+ return 0 2023-10-05T21:08:53.161 INFO:tasks.workunit.client.0.smithi167.stderr:+ expect_fail rbd config global set global debug_ms 10 2023-10-05T21:08:53.161 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd config global set global debug_ms 10 2023-10-05T21:08:53.184 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: not rbd option: debug_ms 2023-10-05T21:08:53.186 INFO:tasks.workunit.client.0.smithi167.stderr:+ return 0 2023-10-05T21:08:53.186 INFO:tasks.workunit.client.0.smithi167.stderr:+ expect_fail rbd config global set global rbd_UNKNOWN false 2023-10-05T21:08:53.186 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd config global set global rbd_UNKNOWN false 2023-10-05T21:08:53.210 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: invalid config key: rbd_UNKNOWN 2023-10-05T21:08:53.211 INFO:tasks.workunit.client.0.smithi167.stderr:+ return 0 2023-10-05T21:08:53.211 INFO:tasks.workunit.client.0.smithi167.stderr:+ expect_fail rbd config global set global rbd_cache INVALID 2023-10-05T21:08:53.211 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd config global set global rbd_cache INVALID 2023-10-05T21:08:53.239 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: error setting rbd_cache: error parsing value: Expected option value to be integer, got 'INVALID' 2023-10-05T21:08:53.242 INFO:tasks.workunit.client.0.smithi167.stderr:+ return 0 2023-10-05T21:08:53.242 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd config global set global rbd_cache false 2023-10-05T21:08:53.274 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd config global set client rbd_cache true 2023-10-05T21:08:53.308 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd config global set client.123 rbd_cache false 2023-10-05T21:08:53.342 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd config global get global rbd_cache 2023-10-05T21:08:53.342 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '^false$' 2023-10-05T21:08:53.371 INFO:tasks.workunit.client.0.smithi167.stdout:false 2023-10-05T21:08:53.372 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd config global get client rbd_cache 2023-10-05T21:08:53.372 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '^true$' 2023-10-05T21:08:53.401 INFO:tasks.workunit.client.0.smithi167.stdout:true 2023-10-05T21:08:53.401 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd config global get client.123 rbd_cache 2023-10-05T21:08:53.402 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '^false$' 2023-10-05T21:08:53.430 INFO:tasks.workunit.client.0.smithi167.stdout:false 2023-10-05T21:08:53.431 INFO:tasks.workunit.client.0.smithi167.stderr:+ expect_fail rbd config global get client.UNKNOWN rbd_cache 2023-10-05T21:08:53.431 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd config global get client.UNKNOWN rbd_cache 2023-10-05T21:08:53.458 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: rbd_cache is not set 2023-10-05T21:08:53.461 INFO:tasks.workunit.client.0.smithi167.stderr:+ return 0 2023-10-05T21:08:53.461 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd config global list global 2023-10-05T21:08:53.461 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '^rbd_cache * false * global *$' 2023-10-05T21:08:53.489 INFO:tasks.workunit.client.0.smithi167.stdout:rbd_cache false global 2023-10-05T21:08:53.489 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd config global list client 2023-10-05T21:08:53.490 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '^rbd_cache * true * client *$' 2023-10-05T21:08:53.518 INFO:tasks.workunit.client.0.smithi167.stdout:rbd_cache true client 2023-10-05T21:08:53.519 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd config global list client.123 2023-10-05T21:08:53.519 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '^rbd_cache * false * client.123 *$' 2023-10-05T21:08:53.547 INFO:tasks.workunit.client.0.smithi167.stdout:rbd_cache false client.123 2023-10-05T21:08:53.548 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd config global list client.UNKNOWN 2023-10-05T21:08:53.548 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '^rbd_cache * true * client *$' 2023-10-05T21:08:53.576 INFO:tasks.workunit.client.0.smithi167.stdout:rbd_cache true client 2023-10-05T21:08:53.576 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd config global rm client rbd_cache 2023-10-05T21:08:53.609 INFO:tasks.workunit.client.0.smithi167.stderr:+ expect_fail rbd config global get client rbd_cache 2023-10-05T21:08:53.609 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd config global get client rbd_cache 2023-10-05T21:08:53.636 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: rbd_cache is not set 2023-10-05T21:08:53.638 INFO:tasks.workunit.client.0.smithi167.stderr:+ return 0 2023-10-05T21:08:53.638 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd config global list client 2023-10-05T21:08:53.638 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '^rbd_cache * false * global *$' 2023-10-05T21:08:53.666 INFO:tasks.workunit.client.0.smithi167.stdout:rbd_cache false global 2023-10-05T21:08:53.667 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd config global rm client.123 rbd_cache 2023-10-05T21:08:53.698 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd config global rm global rbd_cache 2023-10-05T21:08:53.732 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd config pool set rbd rbd_cache true 2023-10-05T21:08:53.774 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd config pool list rbd 2023-10-05T21:08:53.774 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '^rbd_cache * true * pool *$' 2023-10-05T21:08:53.804 INFO:tasks.workunit.client.0.smithi167.stdout:rbd_cache true pool 2023-10-05T21:08:53.804 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd config pool get rbd rbd_cache 2023-10-05T21:08:53.804 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '^true$' 2023-10-05T21:08:53.834 INFO:tasks.workunit.client.0.smithi167.stdout:true 2023-10-05T21:08:53.834 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create -s 1 test1 2023-10-05T21:08:53.861 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:53.859+0000 7f791bcf9540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:53.870 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd config image list rbd/test1 2023-10-05T21:08:53.870 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '^rbd_cache * true * pool *$' 2023-10-05T21:08:53.910 INFO:tasks.workunit.client.0.smithi167.stdout:rbd_cache true pool 2023-10-05T21:08:53.911 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd config image set rbd/test1 rbd_cache false 2023-10-05T21:08:53.954 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd config image list rbd/test1 2023-10-05T21:08:53.955 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '^rbd_cache * false * image *$' 2023-10-05T21:08:53.993 INFO:tasks.workunit.client.0.smithi167.stdout:rbd_cache false image 2023-10-05T21:08:53.993 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd config image get rbd/test1 rbd_cache 2023-10-05T21:08:53.993 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '^false$' 2023-10-05T21:08:54.031 INFO:tasks.workunit.client.0.smithi167.stdout:false 2023-10-05T21:08:54.031 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd config image remove rbd/test1 rbd_cache 2023-10-05T21:08:54.073 INFO:tasks.workunit.client.0.smithi167.stderr:+ expect_fail rbd config image get rbd/test1 rbd_cache 2023-10-05T21:08:54.073 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd config image get rbd/test1 rbd_cache 2023-10-05T21:08:54.106 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: rbd_cache is not set 2023-10-05T21:08:54.111 INFO:tasks.workunit.client.0.smithi167.stderr:+ return 0 2023-10-05T21:08:54.111 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd config image list rbd/test1 2023-10-05T21:08:54.111 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '^rbd_cache * true * pool *$' 2023-10-05T21:08:54.149 INFO:tasks.workunit.client.0.smithi167.stdout:rbd_cache true pool 2023-10-05T21:08:54.149 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd config pool remove rbd rbd_cache 2023-10-05T21:08:54.184 INFO:tasks.workunit.client.0.smithi167.stderr:+ expect_fail rbd config pool get rbd rbd_cache 2023-10-05T21:08:54.184 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd config pool get rbd rbd_cache 2023-10-05T21:08:54.213 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: rbd_cache is not set 2023-10-05T21:08:54.217 INFO:tasks.workunit.client.0.smithi167.stderr:+ return 0 2023-10-05T21:08:54.217 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd config pool list rbd 2023-10-05T21:08:54.217 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '^rbd_cache * true * config *$' 2023-10-05T21:08:54.246 INFO:tasks.workunit.client.0.smithi167.stdout:rbd_cache true config 2023-10-05T21:08:54.247 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test1 2023-10-05T21:08:54.285 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:08:54.288 INFO:tasks.workunit.client.0.smithi167.stderr:+ RBD_CREATE_ARGS= 2023-10-05T21:08:54.288 INFO:tasks.workunit.client.0.smithi167.stderr:+ test_others 2023-10-05T21:08:54.288 INFO:tasks.workunit.client.0.smithi167.stdout:testing import, export, resize, and snapshots... 2023-10-05T21:08:54.289 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'testing import, export, resize, and snapshots...' 2023-10-05T21:08:54.289 INFO:tasks.workunit.client.0.smithi167.stderr:+ TMP_FILES='/tmp/img1 /tmp/img1.new /tmp/img2 /tmp/img2.new /tmp/img3 /tmp/img3.new /tmp/img-diff1.new /tmp/img-diff2.new /tmp/img-diff3.new /tmp/img1.snap1 /tmp/img1.snap1 /tmp/img-diff1.snap1' 2023-10-05T21:08:54.289 INFO:tasks.workunit.client.0.smithi167.stderr:+ remove_images 2023-10-05T21:08:54.289 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:54.371 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:54.448 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:54.523 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:54.599 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:54.878 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:54.964 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:55.041 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:55.119 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:55.197 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:55.274 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:55.351 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:55.433 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:55.517 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:55.594 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:55.669 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:55.745 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:55.822 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:08:55.899 INFO:tasks.workunit.client.0.smithi167.stderr:+ rm -f /tmp/img1 /tmp/img1.new /tmp/img2 /tmp/img2.new /tmp/img3 /tmp/img3.new /tmp/img-diff1.new /tmp/img-diff2.new /tmp/img-diff3.new /tmp/img1.snap1 /tmp/img1.snap1 /tmp/img-diff1.snap1 2023-10-05T21:08:55.899 INFO:tasks.workunit.client.0.smithi167.stderr:+ dd if=/bin/sh of=/tmp/img1 bs=1k count=1 seek=10 2023-10-05T21:08:55.900 INFO:tasks.workunit.client.0.smithi167.stderr:1+0 records in 2023-10-05T21:08:55.900 INFO:tasks.workunit.client.0.smithi167.stderr:1+0 records out 2023-10-05T21:08:55.900 INFO:tasks.workunit.client.0.smithi167.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 3.9601e-05 s, 25.9 MB/s 2023-10-05T21:08:55.900 INFO:tasks.workunit.client.0.smithi167.stderr:+ dd if=/bin/dd of=/tmp/img1 bs=1k count=10 seek=100 2023-10-05T21:08:55.901 INFO:tasks.workunit.client.0.smithi167.stderr:10+0 records in 2023-10-05T21:08:55.901 INFO:tasks.workunit.client.0.smithi167.stderr:10+0 records out 2023-10-05T21:08:55.901 INFO:tasks.workunit.client.0.smithi167.stderr:10240 bytes (10 kB, 10 KiB) copied, 4.9057e-05 s, 209 MB/s 2023-10-05T21:08:55.901 INFO:tasks.workunit.client.0.smithi167.stderr:+ dd if=/bin/rm of=/tmp/img1 bs=1k count=100 seek=1000 2023-10-05T21:08:55.902 INFO:tasks.workunit.client.0.smithi167.stderr:70+1 records in 2023-10-05T21:08:55.902 INFO:tasks.workunit.client.0.smithi167.stderr:70+1 records out 2023-10-05T21:08:55.902 INFO:tasks.workunit.client.0.smithi167.stderr:72208 bytes (72 kB, 71 KiB) copied, 0.00018749 s, 385 MB/s 2023-10-05T21:08:55.902 INFO:tasks.workunit.client.0.smithi167.stderr:+ dd if=/bin/ls of=/tmp/img1 bs=1k seek=10000 2023-10-05T21:08:55.903 INFO:tasks.workunit.client.0.smithi167.stderr:140+1 records in 2023-10-05T21:08:55.903 INFO:tasks.workunit.client.0.smithi167.stderr:140+1 records out 2023-10-05T21:08:55.903 INFO:tasks.workunit.client.0.smithi167.stderr:143376 bytes (143 kB, 140 KiB) copied, 0.000312218 s, 459 MB/s 2023-10-05T21:08:55.903 INFO:tasks.workunit.client.0.smithi167.stderr:+ dd if=/bin/ln of=/tmp/img1 bs=1k seek=100000 2023-10-05T21:08:55.904 INFO:tasks.workunit.client.0.smithi167.stderr:70+1 records in 2023-10-05T21:08:55.904 INFO:tasks.workunit.client.0.smithi167.stderr:70+1 records out 2023-10-05T21:08:55.904 INFO:tasks.workunit.client.0.smithi167.stderr:72312 bytes (72 kB, 71 KiB) copied, 0.000186913 s, 387 MB/s 2023-10-05T21:08:55.904 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd import /tmp/img1 testimg1 2023-10-05T21:08:55.931 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:55.930+0000 7fc5d6462540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:56.012 INFO:tasks.workunit.client.0.smithi167.stderr: Importing image: 4% complete... Importing image: 8% complete... Importing image: 12% complete... Importing image: 16% complete... Importing image: 20% complete... Importing image: 24% complete... Importing image: 28% complete... Importing image: 32% complete... Importing image: 36% complete... Importing image: 40% complete... Importing image: 45% complete... Importing image: 49% complete... Importing image: 53% complete... Importing image: 57% complete... Importing image: 61% complete... Importing image: 65% complete... Importing image: 69% complete... Importing image: 73% complete... Importing image: 77% complete... Importing image: 81% complete... Importing image: 85% complete... Importing image: 90% complete... Importing image: 94% complete... Importing image: 98% complete... Importing image: 100% complete...done. 2023-10-05T21:08:56.015 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd resize testimg1 --size=256 --allow-shrink 2023-10-05T21:08:56.051 INFO:tasks.workunit.client.0.smithi167.stderr: Resizing image: 100% complete...done. 2023-10-05T21:08:56.056 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd export testimg1 /tmp/img2 2023-10-05T21:08:56.153 INFO:tasks.workunit.client.0.smithi167.stderr: Exporting image: 1% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 12% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 23% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 37% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 48% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 62% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 73% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 87% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 98% complete... Exporting image: 100% complete...done. 2023-10-05T21:08:56.156 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap create testimg1 --snap=snap1 2023-10-05T21:08:56.620 INFO:tasks.workunit.client.0.smithi167.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-05T21:08:56.625 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd resize testimg1 --size=128 2023-10-05T21:08:56.657 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: shrinking an image is only allowed with the --allow-shrink flag 2023-10-05T21:08:56.662 INFO:tasks.workunit.client.0.smithi167.stderr:+ true 2023-10-05T21:08:56.662 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd resize testimg1 --size=128 --allow-shrink 2023-10-05T21:08:56.710 INFO:tasks.workunit.client.0.smithi167.stderr: Resizing image: 50% complete... Resizing image: 51% complete... Resizing image: 53% complete... Resizing image: 54% complete... Resizing image: 56% complete... Resizing image: 57% complete... Resizing image: 59% complete... Resizing image: 60% complete... Resizing image: 62% complete... Resizing image: 64% complete... Resizing image: 65% complete... Resizing image: 67% complete... Resizing image: 68% complete... Resizing image: 70% complete... Resizing image: 71% complete... Resizing image: 73% complete... Resizing image: 75% complete... Resizing image: 76% complete... Resizing image: 78% complete... Resizing image: 79% complete... Resizing image: 81% complete... Resizing image: 82% complete... Resizing image: 84% complete... Resizing image: 85% complete... Resizing image: 87% complete... Resizing image: 89% complete... Resizing image: 90% complete... Resizing image: 92% complete... Resizing image: 93% complete... Resizing image: 95% complete... Resizing image: 96% complete... Resizing image: 98% complete... Resizing image: 100% complete...done. 2023-10-05T21:08:56.715 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd export testimg1 /tmp/img3 2023-10-05T21:08:56.784 INFO:tasks.workunit.client.0.smithi167.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2023-10-05T21:08:56.787 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info testimg1 2023-10-05T21:08:56.788 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'size 128 MiB' 2023-10-05T21:08:56.819 INFO:tasks.workunit.client.0.smithi167.stdout: size 128 MiB in 32 objects 2023-10-05T21:08:56.820 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info --snap=snap1 testimg1 2023-10-05T21:08:56.820 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'size 256 MiB' 2023-10-05T21:08:56.852 INFO:tasks.workunit.client.0.smithi167.stdout: size 256 MiB in 64 objects 2023-10-05T21:08:56.852 INFO:tasks.workunit.client.0.smithi167.stderr:+ rm -rf /tmp/diff-testimg1-1 /tmp/diff-testimg1-2 2023-10-05T21:08:56.853 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd export-diff testimg1 --snap=snap1 /tmp/diff-testimg1-1 2023-10-05T21:08:56.947 INFO:tasks.workunit.client.0.smithi167.stderr: Exporting image: 3% complete... Exporting image: 37% complete... Exporting image: 100% complete...done. 2023-10-05T21:08:56.950 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd export-diff testimg1 --from-snap=snap1 /tmp/diff-testimg1-2 2023-10-05T21:08:57.025 INFO:tasks.workunit.client.0.smithi167.stderr: Exporting image: 100% complete...done. 2023-10-05T21:08:57.029 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --size=1 testimg-diff1 2023-10-05T21:08:57.057 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:08:57.055+0000 7f580e8dd540 -1 librbd: Forced V1 image creation. 2023-10-05T21:08:57.065 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd import-diff --sparse-size 8K /tmp/diff-testimg1-1 testimg-diff1 2023-10-05T21:08:57.627 INFO:tasks.workunit.client.0.smithi167.stderr: Importing image diff: 22% complete... Importing image diff: 63% complete... Importing image diff: 99% complete... Importing image diff: 100% complete...done. 2023-10-05T21:08:57.632 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd import-diff --sparse-size 8K /tmp/diff-testimg1-2 testimg-diff1 2023-10-05T21:08:57.692 INFO:tasks.workunit.client.0.smithi167.stderr: Importing image diff: 68% complete... Importing image diff: 96% complete... Importing image diff: 100% complete...done. 2023-10-05T21:08:57.803 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info testimg1 2023-10-05T21:08:57.803 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'size 128 MiB' 2023-10-05T21:08:57.803 INFO:tasks.workunit.client.0.smithi167.stdout: size 128 MiB in 32 objects 2023-10-05T21:08:57.804 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info --snap=snap1 testimg1 2023-10-05T21:08:57.804 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'size 256 MiB' 2023-10-05T21:08:57.804 INFO:tasks.workunit.client.0.smithi167.stdout: size 256 MiB in 64 objects 2023-10-05T21:08:57.805 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info testimg-diff1 2023-10-05T21:08:57.805 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'size 128 MiB' 2023-10-05T21:08:57.805 INFO:tasks.workunit.client.0.smithi167.stdout: size 128 MiB in 32 objects 2023-10-05T21:08:57.805 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info --snap=snap1 testimg-diff1 2023-10-05T21:08:57.805 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'size 256 MiB' 2023-10-05T21:08:57.828 INFO:tasks.workunit.client.0.smithi167.stdout: size 256 MiB in 64 objects 2023-10-05T21:08:57.829 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd copy testimg1 --snap=snap1 testimg2 2023-10-05T21:08:57.913 INFO:tasks.workunit.client.0.smithi167.stderr: Image copy: 1% complete... Image copy: 3% complete... Image copy: 4% complete... Image copy: 6% complete... Image copy: 7% complete... Image copy: 9% complete... Image copy: 10% complete... Image copy: 12% complete... Image copy: 14% complete... Image copy: 15% complete... Image copy: 17% complete... Image copy: 18% complete... Image copy: 20% complete... Image copy: 21% complete... Image copy: 23% complete... Image copy: 25% complete... Image copy: 26% complete... Image copy: 28% complete... Image copy: 29% complete... Image copy: 31% complete... Image copy: 32% complete... Image copy: 34% complete... Image copy: 35% complete... Image copy: 37% complete... Image copy: 39% complete... Image copy: 40% complete... Image copy: 42% complete... Image copy: 43% complete... Image copy: 45% complete... Image copy: 46% complete... Image copy: 48% complete... Image copy: 50% complete... Image copy: 51% complete... Image copy: 53% complete... Image copy: 54% complete... Image copy: 56% complete... Image copy: 57% complete... Image copy: 59% complete... Image copy: 60% complete... Image copy: 62% complete... Image copy: 64% complete... Image copy: 65% complete... Image copy: 67% complete... Image copy: 68% complete... Image copy: 70% complete... Image copy: 71% complete... Image copy: 73% complete... Image copy: 75% complete... Image copy: 76% complete... Image copy: 78% complete... Image copy: 79% complete... Image copy: 81% complete... Image copy: 82% complete... Image copy: 84% complete... Image copy: 85% complete... Image copy: 87% complete... Image copy: 89% complete... Image copy: 90% complete... Image copy: 92% complete... Image copy: 93% complete... Image copy: 95% complete... Image copy: 96% complete... Image copy: 98% complete... Image copy: 100% complete... Image copy: 100% complete...done. 2023-10-05T21:08:57.917 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd copy testimg1 testimg3 2023-10-05T21:08:58.009 INFO:tasks.workunit.client.0.smithi167.stderr: Image copy: 3% complete... Image copy: 6% complete... Image copy: 9% complete... Image copy: 12% complete... Image copy: 15% complete... Image copy: 18% complete... Image copy: 21% complete... Image copy: 25% complete... Image copy: 28% complete... Image copy: 31% complete... Image copy: 34% complete... Image copy: 37% complete... Image copy: 40% complete... Image copy: 43% complete... Image copy: 46% complete... Image copy: 50% complete... Image copy: 53% complete... Image copy: 56% complete... Image copy: 59% complete... Image copy: 62% complete... Image copy: 65% complete... Image copy: 68% complete... Image copy: 71% complete... Image copy: 75% complete... Image copy: 78% complete... Image copy: 81% complete... Image copy: 84% complete... Image copy: 87% complete... Image copy: 90% complete... Image copy: 93% complete... Image copy: 96% complete... Image copy: 100% complete... Image copy: 100% complete...done. 2023-10-05T21:08:58.013 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd copy testimg-diff1 --sparse-size 768K --snap=snap1 testimg-diff2 2023-10-05T21:08:58.136 INFO:tasks.workunit.client.0.smithi167.stderr: Image copy: 1% complete... Image copy: 3% complete... Image copy: 4% complete... Image copy: 6% complete... Image copy: 7% complete... Image copy: 9% complete... Image copy: 10% complete... Image copy: 12% complete... Image copy: 14% complete... Image copy: 15% complete... Image copy: 17% complete... Image copy: 18% complete... Image copy: 20% complete... Image copy: 21% complete... Image copy: 23% complete... Image copy: 25% complete... Image copy: 26% complete... Image copy: 28% complete... Image copy: 29% complete... Image copy: 31% complete... Image copy: 32% complete... Image copy: 34% complete... Image copy: 35% complete... Image copy: 37% complete... Image copy: 39% complete... Image copy: 40% complete... Image copy: 42% complete... Image copy: 43% complete... Image copy: 45% complete... Image copy: 46% complete... Image copy: 48% complete... Image copy: 50% complete... Image copy: 51% complete... Image copy: 53% complete... Image copy: 54% complete... Image copy: 56% complete... Image copy: 57% complete... Image copy: 59% complete... Image copy: 60% complete... Image copy: 62% complete... Image copy: 64% complete... Image copy: 65% complete... Image copy: 67% complete... Image copy: 68% complete... Image copy: 70% complete... Image copy: 71% complete... Image copy: 73% complete... Image copy: 75% complete... Image copy: 76% complete... Image copy: 78% complete... Image copy: 79% complete... Image copy: 81% complete... Image copy: 82% complete... Image copy: 84% complete... Image copy: 85% complete... Image copy: 87% complete... Image copy: 89% complete... Image copy: 90% complete... Image copy: 92% complete... Image copy: 93% complete... Image copy: 95% complete... Image copy: 96% complete... Image copy: 98% complete... Image copy: 100% complete... Image copy: 100% complete...done. 2023-10-05T21:08:58.140 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd copy testimg-diff1 --sparse-size 768K testimg-diff3 2023-10-05T21:08:58.248 INFO:tasks.workunit.client.0.smithi167.stderr: Image copy: 3% complete... Image copy: 6% complete... Image copy: 9% complete... Image copy: 12% complete... Image copy: 15% complete... Image copy: 18% complete... Image copy: 21% complete... Image copy: 25% complete... Image copy: 28% complete... Image copy: 31% complete... Image copy: 34% complete... Image copy: 37% complete... Image copy: 40% complete... Image copy: 43% complete... Image copy: 46% complete... Image copy: 50% complete... Image copy: 53% complete... Image copy: 56% complete... Image copy: 59% complete... Image copy: 62% complete... Image copy: 65% complete... Image copy: 68% complete... Image copy: 71% complete... Image copy: 75% complete... Image copy: 78% complete... Image copy: 81% complete... Image copy: 84% complete... Image copy: 87% complete... Image copy: 90% complete... Image copy: 93% complete... Image copy: 96% complete... Image copy: 100% complete... Image copy: 100% complete...done. 2023-10-05T21:08:58.256 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info testimg2 2023-10-05T21:08:58.256 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'size 256 MiB' 2023-10-05T21:08:58.290 INFO:tasks.workunit.client.0.smithi167.stdout: size 256 MiB in 64 objects 2023-10-05T21:08:58.290 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info testimg3 2023-10-05T21:08:58.291 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'size 128 MiB' 2023-10-05T21:08:58.325 INFO:tasks.workunit.client.0.smithi167.stdout: size 128 MiB in 32 objects 2023-10-05T21:08:58.325 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info testimg-diff2 2023-10-05T21:08:58.325 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'size 256 MiB' 2023-10-05T21:08:58.360 INFO:tasks.workunit.client.0.smithi167.stdout: size 256 MiB in 64 objects 2023-10-05T21:08:58.361 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info testimg-diff3 2023-10-05T21:08:58.361 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'size 128 MiB' 2023-10-05T21:08:58.394 INFO:tasks.workunit.client.0.smithi167.stdout: size 128 MiB in 32 objects 2023-10-05T21:08:58.395 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd deep copy testimg1 testimg4 2023-10-05T21:08:58.709 INFO:tasks.workunit.client.0.smithi167.stderr: Image deep copy: 1% complete... Image deep copy: 3% complete... Image deep copy: 4% complete... Image deep copy: 6% complete... Image deep copy: 7% complete... Image deep copy: 9% complete... Image deep copy: 10% complete... Image deep copy: 12% complete... Image deep copy: 14% complete... Image deep copy: 15% complete... Image deep copy: 17% complete... Image deep copy: 18% complete... Image deep copy: 20% complete... Image deep copy: 21% complete... Image deep copy: 23% complete... Image deep copy: 25% complete... Image deep copy: 26% complete... Image deep copy: 28% complete... Image deep copy: 29% complete... Image deep copy: 31% complete... Image deep copy: 32% complete... Image deep copy: 34% complete... Image deep copy: 35% complete... Image deep copy: 37% complete... Image deep copy: 39% complete... Image deep copy: 40% complete... Image deep copy: 42% complete... Image deep copy: 43% complete... Image deep copy: 45% complete... Image deep copy: 46% complete... Image deep copy: 48% complete... Image deep copy: 50% complete... Image deep copy: 51% complete... Image deep copy: 53% complete... Image deep copy: 54% complete... Image deep copy: 56% complete... Image deep copy: 57% complete... Image deep copy: 59% complete... Image deep copy: 60% complete... Image deep copy: 62% complete... Image deep copy: 64% complete... Image deep copy: 65% complete... Image deep copy: 67% complete... Image deep copy: 68% complete... Image deep copy: 70% complete... Image deep copy: 71% complete... Image deep copy: 73% complete... Image deep copy: 75% complete... Image deep copy: 76% complete... Image deep copy: 78% complete... Image deep copy: 79% complete... Image deep copy: 81% complete... Image deep copy: 82% complete... Image deep copy: 84% complete... Image deep copy: 85% complete... Image deep copy: 87% complete... Image deep copy: 89% complete... Image deep copy: 90% complete... Image deep copy: 92% complete... Image deep copy: 93% complete... Image deep copy: 95% complete... Image deep copy: 96% complete... Image deep copy: 98% complete... Image deep copy: 100% complete...2023-10-05T21:08:58.707+0000 7f48145cf700 0 -- 172.21.15.167:0/3796559088 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x7f47f4001be0 msgr2=0x7f47f4004090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:08:58.712 INFO:tasks.workunit.client.0.smithi167.stderr: Image deep copy: 100% complete...done. 2023-10-05T21:08:58.715 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd deep copy testimg1 --snap=snap1 testimg5 2023-10-05T21:08:59.712 INFO:tasks.workunit.client.0.smithi167.stderr: Image deep copy: 1% complete... Image deep copy: 3% complete... Image deep copy: 4% complete... Image deep copy: 6% complete... Image deep copy: 7% complete... Image deep copy: 9% complete... Image deep copy: 10% complete... Image deep copy: 12% complete... Image deep copy: 14% complete... Image deep copy: 15% complete... Image deep copy: 17% complete... Image deep copy: 18% complete... Image deep copy: 20% complete... Image deep copy: 21% complete... Image deep copy: 23% complete... Image deep copy: 25% complete... Image deep copy: 26% complete... Image deep copy: 28% complete... Image deep copy: 29% complete... Image deep copy: 31% complete... Image deep copy: 32% complete... Image deep copy: 34% complete... Image deep copy: 35% complete... Image deep copy: 37% complete... Image deep copy: 39% complete... Image deep copy: 40% complete... Image deep copy: 42% complete... Image deep copy: 43% complete... Image deep copy: 45% complete... Image deep copy: 46% complete... Image deep copy: 48% complete... Image deep copy: 50% complete... Image deep copy: 51% complete... Image deep copy: 53% complete... Image deep copy: 54% complete... Image deep copy: 56% complete... Image deep copy: 57% complete... Image deep copy: 59% complete... Image deep copy: 60% complete... Image deep copy: 62% complete... Image deep copy: 64% complete... Image deep copy: 65% complete... Image deep copy: 67% complete... Image deep copy: 68% complete... Image deep copy: 70% complete... Image deep copy: 71% complete... Image deep copy: 73% complete... Image deep copy: 75% complete... Image deep copy: 76% complete... Image deep copy: 78% complete... Image deep copy: 79% complete... Image deep copy: 81% complete... Image deep copy: 82% complete... Image deep copy: 84% complete... Image deep copy: 85% complete... Image deep copy: 87% complete... Image deep copy: 89% complete... Image deep copy: 90% complete... Image deep copy: 92% complete... Image deep copy: 93% complete... Image deep copy: 95% complete... Image deep copy: 96% complete... Image deep copy: 98% complete... Image deep copy: 100% complete... Image deep copy: 100% complete...done. 2023-10-05T21:08:59.716 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info testimg4 2023-10-05T21:08:59.716 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'size 128 MiB' 2023-10-05T21:08:59.752 INFO:tasks.workunit.client.0.smithi167.stdout: size 128 MiB in 32 objects 2023-10-05T21:08:59.752 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info testimg5 2023-10-05T21:08:59.752 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'size 256 MiB' 2023-10-05T21:08:59.791 INFO:tasks.workunit.client.0.smithi167.stdout: size 256 MiB in 64 objects 2023-10-05T21:08:59.792 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap ls testimg4 2023-10-05T21:08:59.792 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep -v SNAPID 2023-10-05T21:08:59.792 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:08:59.792 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 1 2023-10-05T21:08:59.826 INFO:tasks.workunit.client.0.smithi167.stdout:1 2023-10-05T21:08:59.827 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap ls testimg4 2023-10-05T21:08:59.827 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '.*snap1.*' 2023-10-05T21:08:59.860 INFO:tasks.workunit.client.0.smithi167.stdout: 18 snap1 256 MiB Thu Oct 5 21:08:58 2023 2023-10-05T21:08:59.861 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd export testimg1 /tmp/img1.new 2023-10-05T21:08:59.926 INFO:tasks.workunit.client.0.smithi167.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2023-10-05T21:08:59.930 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd export testimg2 /tmp/img2.new 2023-10-05T21:09:00.028 INFO:tasks.workunit.client.0.smithi167.stderr: Exporting image: 1% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 12% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 23% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 37% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 48% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 62% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 73% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 87% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 98% complete... Exporting image: 100% complete...done. 2023-10-05T21:09:00.032 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd export testimg3 /tmp/img3.new 2023-10-05T21:09:00.095 INFO:tasks.workunit.client.0.smithi167.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2023-10-05T21:09:00.098 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd export testimg-diff1 /tmp/img-diff1.new 2023-10-05T21:09:00.167 INFO:tasks.workunit.client.0.smithi167.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2023-10-05T21:09:00.170 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd export testimg-diff2 /tmp/img-diff2.new 2023-10-05T21:09:00.275 INFO:tasks.workunit.client.0.smithi167.stderr: Exporting image: 1% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 12% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 23% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 37% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 48% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 62% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 73% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 87% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 98% complete... Exporting image: 100% complete...done. 2023-10-05T21:09:00.278 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd export testimg-diff3 /tmp/img-diff3.new 2023-10-05T21:09:00.349 INFO:tasks.workunit.client.0.smithi167.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2023-10-05T21:09:00.351 INFO:tasks.workunit.client.0.smithi167.stderr:+ cmp /tmp/img2 /tmp/img2.new 2023-10-05T21:09:00.547 INFO:tasks.workunit.client.0.smithi167.stderr:+ cmp /tmp/img3 /tmp/img3.new 2023-10-05T21:09:00.643 INFO:tasks.workunit.client.0.smithi167.stderr:+ cmp /tmp/img2 /tmp/img-diff2.new 2023-10-05T21:09:00.803 INFO:tasks.workunit.client.0.smithi167.stderr:+ cmp /tmp/img3 /tmp/img-diff3.new 2023-10-05T21:09:00.883 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap rollback --snap=snap1 testimg1 2023-10-05T21:09:00.979 INFO:tasks.workunit.client.0.smithi167.stderr: Rolling back to snapshot: 1% complete... Rolling back to snapshot: 3% complete... Rolling back to snapshot: 4% complete... Rolling back to snapshot: 6% complete... Rolling back to snapshot: 7% complete... Rolling back to snapshot: 9% complete... Rolling back to snapshot: 10% complete... Rolling back to snapshot: 12% complete... Rolling back to snapshot: 14% complete... Rolling back to snapshot: 15% complete... Rolling back to snapshot: 17% complete... Rolling back to snapshot: 18% complete... Rolling back to snapshot: 20% complete... Rolling back to snapshot: 21% complete... Rolling back to snapshot: 23% complete... Rolling back to snapshot: 25% complete... Rolling back to snapshot: 26% complete... Rolling back to snapshot: 28% complete... Rolling back to snapshot: 29% complete... Rolling back to snapshot: 31% complete... Rolling back to snapshot: 32% complete... Rolling back to snapshot: 34% complete... Rolling back to snapshot: 35% complete... Rolling back to snapshot: 37% complete... Rolling back to snapshot: 39% complete... Rolling back to snapshot: 40% complete... Rolling back to snapshot: 42% complete... Rolling back to snapshot: 43% complete... Rolling back to snapshot: 45% complete... Rolling back to snapshot: 46% complete... Rolling back to snapshot: 48% complete... Rolling back to snapshot: 50% complete... Rolling back to snapshot: 51% complete... Rolling back to snapshot: 53% complete... Rolling back to snapshot: 54% complete... Rolling back to snapshot: 56% complete... Rolling back to snapshot: 57% complete... Rolling back to snapshot: 59% complete... Rolling back to snapshot: 60% complete... Rolling back to snapshot: 62% complete... Rolling back to snapshot: 64% complete... Rolling back to snapshot: 65% complete... Rolling back to snapshot: 67% complete... Rolling back to snapshot: 68% complete... Rolling back to snapshot: 70% complete... Rolling back to snapshot: 71% complete... Rolling back to snapshot: 73% complete... Rolling back to snapshot: 75% complete... Rolling back to snapshot: 76% complete... Rolling back to snapshot: 78% complete... Rolling back to snapshot: 79% complete... Rolling back to snapshot: 81% complete... Rolling back to snapshot: 82% complete... Rolling back to snapshot: 84% complete... Rolling back to snapshot: 85% complete... Rolling back to snapshot: 87% complete... Rolling back to snapshot: 89% complete... Rolling back to snapshot: 90% complete... Rolling back to snapshot: 92% complete... Rolling back to snapshot: 93% complete... Rolling back to snapshot: 95% complete... Rolling back to snapshot: 96% complete... Rolling back to snapshot: 98% complete... Rolling back to snapshot: 100% complete...done. 2023-10-05T21:09:00.985 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap rollback --snap=snap1 testimg-diff1 2023-10-05T21:09:01.080 INFO:tasks.workunit.client.0.smithi167.stderr: Rolling back to snapshot: 1% complete... Rolling back to snapshot: 3% complete... Rolling back to snapshot: 4% complete... Rolling back to snapshot: 6% complete... Rolling back to snapshot: 7% complete... Rolling back to snapshot: 9% complete... Rolling back to snapshot: 10% complete... Rolling back to snapshot: 12% complete... Rolling back to snapshot: 14% complete... Rolling back to snapshot: 15% complete... Rolling back to snapshot: 17% complete... Rolling back to snapshot: 18% complete... Rolling back to snapshot: 20% complete... Rolling back to snapshot: 21% complete... Rolling back to snapshot: 23% complete... Rolling back to snapshot: 25% complete... Rolling back to snapshot: 26% complete... Rolling back to snapshot: 28% complete... Rolling back to snapshot: 29% complete... Rolling back to snapshot: 31% complete... Rolling back to snapshot: 32% complete... Rolling back to snapshot: 34% complete... Rolling back to snapshot: 35% complete... Rolling back to snapshot: 37% complete... Rolling back to snapshot: 39% complete... Rolling back to snapshot: 40% complete... Rolling back to snapshot: 42% complete... Rolling back to snapshot: 43% complete... Rolling back to snapshot: 45% complete... Rolling back to snapshot: 46% complete... Rolling back to snapshot: 48% complete... Rolling back to snapshot: 50% complete... Rolling back to snapshot: 51% complete... Rolling back to snapshot: 53% complete... Rolling back to snapshot: 54% complete... Rolling back to snapshot: 56% complete... Rolling back to snapshot: 57% complete... Rolling back to snapshot: 59% complete... Rolling back to snapshot: 60% complete... Rolling back to snapshot: 62% complete... Rolling back to snapshot: 64% complete... Rolling back to snapshot: 65% complete... Rolling back to snapshot: 67% complete... Rolling back to snapshot: 68% complete... Rolling back to snapshot: 70% complete... Rolling back to snapshot: 71% complete... Rolling back to snapshot: 73% complete... Rolling back to snapshot: 75% complete... Rolling back to snapshot: 76% complete... Rolling back to snapshot: 78% complete... Rolling back to snapshot: 79% complete... Rolling back to snapshot: 81% complete... Rolling back to snapshot: 82% complete... Rolling back to snapshot: 84% complete... Rolling back to snapshot: 85% complete... Rolling back to snapshot: 87% complete... Rolling back to snapshot: 89% complete... Rolling back to snapshot: 90% complete... Rolling back to snapshot: 92% complete... Rolling back to snapshot: 93% complete... Rolling back to snapshot: 95% complete... Rolling back to snapshot: 96% complete... Rolling back to snapshot: 98% complete... Rolling back to snapshot: 100% complete...done. 2023-10-05T21:09:01.087 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info testimg1 2023-10-05T21:09:01.087 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'size 256 MiB' 2023-10-05T21:09:01.121 INFO:tasks.workunit.client.0.smithi167.stdout: size 256 MiB in 64 objects 2023-10-05T21:09:01.121 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info testimg-diff1 2023-10-05T21:09:01.122 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'size 256 MiB' 2023-10-05T21:09:01.153 INFO:tasks.workunit.client.0.smithi167.stdout: size 256 MiB in 64 objects 2023-10-05T21:09:01.154 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd export testimg1 /tmp/img1.snap1 2023-10-05T21:09:01.257 INFO:tasks.workunit.client.0.smithi167.stderr: Exporting image: 1% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 12% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 23% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 37% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 48% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 62% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 73% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 87% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 98% complete... Exporting image: 100% complete...done. 2023-10-05T21:09:01.260 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd export testimg-diff1 /tmp/img-diff1.snap1 2023-10-05T21:09:01.349 INFO:tasks.workunit.client.0.smithi167.stderr: Exporting image: 1% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 12% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 23% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 37% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 48% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 62% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 73% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 87% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 98% complete... Exporting image: 100% complete...done. 2023-10-05T21:09:01.353 INFO:tasks.workunit.client.0.smithi167.stderr:+ cmp /tmp/img2 /tmp/img1.snap1 2023-10-05T21:09:01.503 INFO:tasks.workunit.client.0.smithi167.stderr:+ cmp /tmp/img2 /tmp/img-diff1.snap1 2023-10-05T21:09:01.654 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm testimg2 2023-10-05T21:09:01.742 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 1% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 12% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 23% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 37% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 48% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 62% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 73% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 87% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 95% complete...2023-10-05T21:09:01.740+0000 7f97b8bd3700 0 -- 172.21.15.167:0/230372705 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x55df374664d0 msgr2=0x55df37477990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:09:01.769 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 96% complete... Removing image: 98% complete...2023-10-05T21:09:01.767+0000 7f97b83d2700 0 -- 172.21.15.167:0/230372705 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x55df3750f830 msgr2=0x55df374bfb40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-05T21:09:01.769 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:09:01.772 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm testimg3 2023-10-05T21:09:01.867 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2023-10-05T21:09:01.869 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create testimg2 -s 0 2023-10-05T21:09:01.898 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:09:01.896+0000 7fe43da8c540 -1 librbd: Forced V1 image creation. 2023-10-05T21:09:01.905 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd cp testimg2 testimg3 2023-10-05T21:09:01.964 INFO:tasks.workunit.client.0.smithi167.stderr: Image copy: 100% complete...done. 2023-10-05T21:09:01.967 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd deep cp testimg2 testimg6 2023-10-05T21:09:02.021 INFO:tasks.workunit.client.0.smithi167.stderr: Image deep copy: 100% complete...done. 2023-10-05T21:09:02.024 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap rm --snap=snap1 testimg1 2023-10-05T21:09:02.648 INFO:tasks.workunit.client.0.smithi167.stderr: Removing snap: 100% complete...done. 2023-10-05T21:09:02.653 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap rm --snap=snap1 testimg-diff1 2023-10-05T21:09:03.652 INFO:tasks.workunit.client.0.smithi167.stderr: Removing snap: 100% complete...done. 2023-10-05T21:09:03.657 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info --snap=snap1 testimg1 2023-10-05T21:09:03.658 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'error setting snapshot context: (2) No such file or directory' 2023-10-05T21:09:03.693 INFO:tasks.workunit.client.0.smithi167.stdout:error setting snapshot context: (2) No such file or directory 2023-10-05T21:09:03.694 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info --snap=snap1 testimg-diff1 2023-10-05T21:09:03.694 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'error setting snapshot context: (2) No such file or directory' 2023-10-05T21:09:03.729 INFO:tasks.workunit.client.0.smithi167.stdout:error setting snapshot context: (2) No such file or directory 2023-10-05T21:09:03.730 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd sparsify testimg1 2023-10-05T21:09:03.848 INFO:tasks.workunit.client.0.smithi167.stderr: Image sparsify: 1% complete... Image sparsify: 3% complete... Image sparsify: 4% complete... Image sparsify: 6% complete... Image sparsify: 7% complete... Image sparsify: 9% complete... Image sparsify: 10% complete... Image sparsify: 12% complete... Image sparsify: 14% complete... Image sparsify: 15% complete... Image sparsify: 17% complete... Image sparsify: 18% complete... Image sparsify: 20% complete... Image sparsify: 21% complete... Image sparsify: 23% complete... Image sparsify: 25% complete... Image sparsify: 26% complete... Image sparsify: 28% complete... Image sparsify: 29% complete... Image sparsify: 31% complete... Image sparsify: 32% complete... Image sparsify: 34% complete... Image sparsify: 35% complete... Image sparsify: 37% complete... Image sparsify: 39% complete... Image sparsify: 40% complete... Image sparsify: 42% complete... Image sparsify: 43% complete... Image sparsify: 45% complete... Image sparsify: 46% complete... Image sparsify: 48% complete... Image sparsify: 50% complete... Image sparsify: 51% complete... Image sparsify: 53% complete... Image sparsify: 54% complete... Image sparsify: 56% complete... Image sparsify: 57% complete... Image sparsify: 59% complete... Image sparsify: 60% complete... Image sparsify: 62% complete... Image sparsify: 64% complete... Image sparsify: 65% complete... Image sparsify: 67% complete... Image sparsify: 68% complete... Image sparsify: 70% complete... Image sparsify: 71% complete... Image sparsify: 73% complete... Image sparsify: 75% complete... Image sparsify: 76% complete... Image sparsify: 78% complete... Image sparsify: 79% complete... Image sparsify: 81% complete... Image sparsify: 82% complete... Image sparsify: 84% complete... Image sparsify: 85% complete... Image sparsify: 87% complete... Image sparsify: 89% complete... Image sparsify: 90% complete... Image sparsify: 92% complete... Image sparsify: 93% complete... Image sparsify: 95% complete... Image sparsify: 96% complete... Image sparsify: 98% complete... Image sparsify: 100% complete...done. 2023-10-05T21:09:03.853 INFO:tasks.workunit.client.0.smithi167.stderr:+ remove_images 2023-10-05T21:09:03.853 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:03.968 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:04.046 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:04.167 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:04.757 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:05.777 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:05.904 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:06.012 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:06.168 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:06.312 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:06.395 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:06.479 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:06.557 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:06.839 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:06.917 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:06.994 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:07.071 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:07.151 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:07.230 INFO:tasks.workunit.client.0.smithi167.stderr:+ rm -f /tmp/img1 /tmp/img1.new /tmp/img2 /tmp/img2.new /tmp/img3 /tmp/img3.new /tmp/img-diff1.new /tmp/img-diff2.new /tmp/img-diff3.new /tmp/img1.snap1 /tmp/img1.snap1 /tmp/img-diff1.snap1 2023-10-05T21:09:07.339 INFO:tasks.workunit.client.0.smithi167.stderr:+ test_locking 2023-10-05T21:09:07.339 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'testing locking...' 2023-10-05T21:09:07.340 INFO:tasks.workunit.client.0.smithi167.stdout:testing locking... 2023-10-05T21:09:07.340 INFO:tasks.workunit.client.0.smithi167.stderr:+ remove_images 2023-10-05T21:09:07.340 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:07.417 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:07.494 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:07.574 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:07.652 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:07.731 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:07.810 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:07.890 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:07.974 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:08.052 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:08.129 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:08.206 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:08.284 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:08.361 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:08.438 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:08.717 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:08.797 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:08.875 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:08.954 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create -s 1 test1 2023-10-05T21:09:08.982 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:09:08.980+0000 7f79166ea540 -1 librbd: Forced V1 image creation. 2023-10-05T21:09:08.990 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd lock list test1 2023-10-05T21:09:08.990 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:09:08.990 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '^0$' 2023-10-05T21:09:09.021 INFO:tasks.workunit.client.0.smithi167.stdout:0 2023-10-05T21:09:09.022 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd lock add test1 id 2023-10-05T21:09:09.060 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd lock list test1 2023-10-05T21:09:09.060 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep ' 1 ' 2023-10-05T21:09:09.091 INFO:tasks.workunit.client.0.smithi167.stdout:There is 1 exclusive lock on this image. 2023-10-05T21:09:09.092 INFO:tasks.workunit.client.0.smithi167.stderr:++ rbd lock list test1 2023-10-05T21:09:09.092 INFO:tasks.workunit.client.0.smithi167.stderr:++ tail -n 1 2023-10-05T21:09:09.092 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1;}' 2023-10-05T21:09:09.125 INFO:tasks.workunit.client.0.smithi167.stderr:+ LOCKER=client.7784 2023-10-05T21:09:09.125 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd lock remove test1 id client.7784 2023-10-05T21:09:09.679 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd lock list test1 2023-10-05T21:09:09.679 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:09:09.679 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '^0$' 2023-10-05T21:09:09.713 INFO:tasks.workunit.client.0.smithi167.stdout:0 2023-10-05T21:09:09.713 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd lock add test1 id --shared tag 2023-10-05T21:09:09.757 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd lock list test1 2023-10-05T21:09:09.757 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep ' 1 ' 2023-10-05T21:09:09.790 INFO:tasks.workunit.client.0.smithi167.stdout:There is 1 shared lock on this image. 2023-10-05T21:09:09.790 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd lock add test1 id --shared tag 2023-10-05T21:09:09.830 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd lock list test1 2023-10-05T21:09:09.830 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep ' 2 ' 2023-10-05T21:09:09.861 INFO:tasks.workunit.client.0.smithi167.stdout:There are 2 shared locks on this image. 2023-10-05T21:09:09.862 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd lock add test1 id2 --shared tag 2023-10-05T21:09:09.903 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd lock list test1 2023-10-05T21:09:09.903 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep ' 3 ' 2023-10-05T21:09:09.934 INFO:tasks.workunit.client.0.smithi167.stdout:There are 3 shared locks on this image. 2023-10-05T21:09:09.935 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd lock list test1 2023-10-05T21:09:09.935 INFO:tasks.workunit.client.0.smithi167.stderr:+ tail -n 1 2023-10-05T21:09:09.935 INFO:tasks.workunit.client.0.smithi167.stderr:+ awk '{print $2, $1;}' 2023-10-05T21:09:09.935 INFO:tasks.workunit.client.0.smithi167.stderr:+ xargs rbd lock remove test1 2023-10-05T21:09:10.687 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info test1 2023-10-05T21:09:10.687 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep -qE 'features:.*exclusive' 2023-10-05T21:09:10.724 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test1 2023-10-05T21:09:10.768 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:09:10.771 INFO:tasks.workunit.client.0.smithi167.stderr:+ test_thick_provision 2023-10-05T21:09:10.771 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'testing thick provision...' 2023-10-05T21:09:10.772 INFO:tasks.workunit.client.0.smithi167.stdout:testing thick provision... 2023-10-05T21:09:10.772 INFO:tasks.workunit.client.0.smithi167.stderr:+ remove_images 2023-10-05T21:09:10.772 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:10.852 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:10.930 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:11.047 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:11.083 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:11.162 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:11.240 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:11.319 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:11.403 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:11.482 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:11.594 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:11.643 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:11.721 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:11.801 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:11.880 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:12.157 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:12.234 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:12.311 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:12.388 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --thick-provision -s 64M test1 2023-10-05T21:09:12.416 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:09:12.415+0000 7fdfcdef4540 -1 librbd: Forced V1 image creation. 2023-10-05T21:09:12.779 INFO:tasks.workunit.client.0.smithi167.stderr: Thick provisioning: 6% complete... Thick provisioning: 12% complete... Thick provisioning: 18% complete... Thick provisioning: 25% complete... Thick provisioning: 31% complete... Thick provisioning: 37% complete... Thick provisioning: 43% complete... Thick provisioning: 50% complete... Thick provisioning: 56% complete... Thick provisioning: 62% complete... Thick provisioning: 68% complete... Thick provisioning: 75% complete... Thick provisioning: 81% complete... Thick provisioning: 87% complete... Thick provisioning: 93% complete... Thick provisioning: 100% complete... Thick provisioning: 100% complete...done. 2023-10-05T21:09:12.785 INFO:tasks.workunit.client.0.smithi167.stderr:+ count=0 2023-10-05T21:09:12.785 INFO:tasks.workunit.client.0.smithi167.stderr:+ ret= 2023-10-05T21:09:12.785 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 0 -lt 10 ']' 2023-10-05T21:09:12.785 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd du 2023-10-05T21:09:12.785 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep test1 2023-10-05T21:09:12.785 INFO:tasks.workunit.client.0.smithi167.stderr:+ tr -s ' ' 2023-10-05T21:09:12.785 INFO:tasks.workunit.client.0.smithi167.stderr:+ cut -d ' ' -f 4-5 2023-10-05T21:09:12.786 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '^64 MiB' 2023-10-05T21:09:13.007 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:09:13.006+0000 7f4e88b3b700 0 --2- 172.21.15.167:0/363876920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55956b254150 0x55956b2dc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2023-10-05T21:09:13.014 INFO:tasks.workunit.client.0.smithi167.stderr:warning: fast-diff map is not enabled for test1. operation may be slow. 2023-10-05T21:09:13.019 INFO:tasks.workunit.client.0.smithi167.stdout:64 MiB 2023-10-05T21:09:13.019 INFO:tasks.workunit.client.0.smithi167.stderr:+ ret=0 2023-10-05T21:09:13.020 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 0 = 0 ']' 2023-10-05T21:09:13.020 INFO:tasks.workunit.client.0.smithi167.stderr:+ break 2023-10-05T21:09:13.020 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd du 2023-10-05T21:09:13.048 INFO:tasks.workunit.client.0.smithi167.stderr:warning: fast-diff map is not enabled for test1. operation may be slow. 2023-10-05T21:09:13.049 INFO:tasks.workunit.client.0.smithi167.stdout:NAME PROVISIONED USED 2023-10-05T21:09:13.050 INFO:tasks.workunit.client.0.smithi167.stdout:test1 64 MiB 64 MiB 2023-10-05T21:09:13.052 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 0 '!=' 0 ']' 2023-10-05T21:09:13.052 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test1 2023-10-05T21:09:13.115 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 6% complete... Removing image: 12% complete... Removing image: 18% complete... Removing image: 25% complete... Removing image: 31% complete... Removing image: 37% complete... Removing image: 43% complete... Removing image: 50% complete... Removing image: 56% complete... Removing image: 62% complete... Removing image: 68% complete... Removing image: 75% complete... Removing image: 81% complete... Removing image: 87% complete... Removing image: 93% complete... Removing image: 100% complete...done. 2023-10-05T21:09:13.120 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls 2023-10-05T21:09:13.120 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep test1 2023-10-05T21:09:13.121 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '^0$' 2023-10-05T21:09:13.121 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:09:13.344 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:09:13.342+0000 7f4fa04c2700 0 --2- 172.21.15.167:0/2855005743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5584e26833e0 0x5584e2681960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2023-10-05T21:09:13.350 INFO:tasks.workunit.client.0.smithi167.stdout:0 2023-10-05T21:09:13.350 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --thick-provision -s 4G test1 2023-10-05T21:09:13.378 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:09:13.376+0000 7f58e043f540 -1 librbd: Forced V1 image creation. 2023-10-05T21:09:14.646 INFO:tasks.workunit.client.0.smithi167.stderr: Thick provisioning: 1% complete... Thick provisioning: 2% complete... Thick provisioning: 3% complete... Thick provisioning: 4% complete... Thick provisioning: 5% complete... Thick provisioning: 6% complete... Thick provisioning: 7% complete... Thick provisioning: 8% complete... Thick provisioning: 9% complete... Thick provisioning: 10% complete... Thick provisioning: 11% complete...2023-10-05T21:09:14.645+0000 7f58d7f30700 0 -- 172.21.15.167:0/110706716 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x55752d2910e0 msgr2=0x55752d2934c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:09:14.881 INFO:tasks.workunit.client.0.smithi167.stderr: Thick provisioning: 12% complete... Thick provisioning: 13% complete...2023-10-05T21:09:14.879+0000 7f58d8731700 0 -- 172.21.15.167:0/110706716 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x55752d1ec350 msgr2=0x55752d1e2f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:09:26.144 INFO:tasks.workunit.client.0.smithi167.stderr: Thick provisioning: 14% complete... Thick provisioning: 15% complete... Thick provisioning: 16% complete... Thick provisioning: 17% complete... Thick provisioning: 18% complete... Thick provisioning: 19% complete... Thick provisioning: 20% complete... Thick provisioning: 21% complete... Thick provisioning: 22% complete... Thick provisioning: 23% complete... Thick provisioning: 24% complete... Thick provisioning: 25% complete... Thick provisioning: 26% complete... Thick provisioning: 27% complete... Thick provisioning: 28% complete... Thick provisioning: 29% complete... Thick provisioning: 30% complete... Thick provisioning: 31% complete... Thick provisioning: 32% complete... Thick provisioning: 33% complete... Thick provisioning: 34% complete... Thick provisioning: 35% complete... Thick provisioning: 36% complete... Thick provisioning: 37% complete... Thick provisioning: 38% complete... Thick provisioning: 39% complete... Thick provisioning: 40% complete... Thick provisioning: 41% complete... Thick provisioning: 42% complete... Thick provisioning: 43% complete... Thick provisioning: 44% complete... Thick provisioning: 45% complete... Thick provisioning: 46% complete... Thick provisioning: 47% complete... Thick provisioning: 48% complete... Thick provisioning: 49% complete... Thick provisioning: 50% complete... Thick provisioning: 51% complete... Thick provisioning: 52% complete... Thick provisioning: 53% complete... Thick provisioning: 54% complete... Thick provisioning: 55% complete... Thick provisioning: 56% complete... Thick provisioning: 57% complete... Thick provisioning: 58% complete... Thick provisioning: 59% complete... Thick provisioning: 60% complete... Thick provisioning: 61% complete... Thick provisioning: 62% complete... Thick provisioning: 63% complete... Thick provisioning: 64% complete... Thick provisioning: 65% complete... Thick provisioning: 66% complete... Thick provisioning: 67% complete... Thick provisioning: 68% complete... Thick provisioning: 69% complete... Thick provisioning: 70% complete... Thick provisioning: 71% complete... Thick provisioning: 72% complete... Thick provisioning: 73% complete... Thick provisioning: 74% complete... Thick provisioning: 75% complete... Thick provisioning: 76% complete... Thick provisioning: 77% complete... Thick provisioning: 78% complete... Thick provisioning: 79% complete... Thick provisioning: 80% complete... Thick provisioning: 81% complete... Thick provisioning: 82% complete... Thick provisioning: 83% complete... Thick provisioning: 84% complete... Thick provisioning: 85% complete... Thick provisioning: 86% complete... Thick provisioning: 87% complete... Thick provisioning: 88% complete... Thick provisioning: 89% complete... Thick provisioning: 90% complete... Thick provisioning: 91% complete... Thick provisioning: 92% complete... Thick provisioning: 93% complete... Thick provisioning: 94% complete... Thick provisioning: 95% complete... Thick provisioning: 96% complete... Thick provisioning: 97% complete... Thick provisioning: 98% complete... Thick provisioning: 99% complete... Thick provisioning: 100% complete... Thick provisioning: 100% complete...done. 2023-10-05T21:09:26.153 INFO:tasks.workunit.client.0.smithi167.stderr:+ count=0 2023-10-05T21:09:26.154 INFO:tasks.workunit.client.0.smithi167.stderr:+ ret= 2023-10-05T21:09:26.154 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 0 -lt 10 ']' 2023-10-05T21:09:26.154 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd du 2023-10-05T21:09:26.154 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep test1 2023-10-05T21:09:26.154 INFO:tasks.workunit.client.0.smithi167.stderr:+ tr -s ' ' 2023-10-05T21:09:26.154 INFO:tasks.workunit.client.0.smithi167.stderr:+ cut -d ' ' -f 4-5 2023-10-05T21:09:26.154 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '^4 GiB' 2023-10-05T21:09:26.182 INFO:tasks.workunit.client.0.smithi167.stderr:warning: fast-diff map is not enabled for test1. operation may be slow. 2023-10-05T21:09:26.221 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:09:26.220+0000 7f9915975700 0 -- 172.21.15.167:0/4280843802 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x55714c5e61f0 msgr2=0x55714c5e85d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:09:26.250 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:09:26.249+0000 7f9915975700 0 -- 172.21.15.167:0/4280843802 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f98f803a440 msgr2=0x7f98f803c860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-05T21:09:26.367 INFO:tasks.workunit.client.0.smithi167.stdout:4 GiB 2023-10-05T21:09:26.368 INFO:tasks.workunit.client.0.smithi167.stderr:+ ret=0 2023-10-05T21:09:26.368 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 0 = 0 ']' 2023-10-05T21:09:26.368 INFO:tasks.workunit.client.0.smithi167.stderr:+ break 2023-10-05T21:09:26.368 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd du 2023-10-05T21:09:26.396 INFO:tasks.workunit.client.0.smithi167.stderr:warning: fast-diff map is not enabled for test1. operation may be slow. 2023-10-05T21:09:26.407 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:09:26.406+0000 7f052b4e4700 0 -- 172.21.15.167:0/4215091786 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x56415c201440 msgr2=0x56415c203820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:09:26.410 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:09:26.409+0000 7f052bce5700 0 -- 172.21.15.167:0/4215091786 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x56415c15ba60 msgr2=0x56415c152180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:09:26.535 INFO:tasks.workunit.client.0.smithi167.stdout:NAME PROVISIONED USED 2023-10-05T21:09:26.535 INFO:tasks.workunit.client.0.smithi167.stdout:test1 4 GiB 4 GiB 2023-10-05T21:09:26.538 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 0 '!=' 0 ']' 2023-10-05T21:09:26.538 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test1 2023-10-05T21:09:26.652 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete...2023-10-05T21:09:26.651+0000 7fd5f2de5700 0 -- 172.21.15.167:0/872963376 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x55c63a711890 msgr2=0x55c63a7b68d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:09:26.672 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 11% complete... Removing image: 12% complete...2023-10-05T21:09:26.671+0000 7fd5f2de5700 0 -- 172.21.15.167:0/872963376 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x7fd5d403a380 msgr2=0x7fd5d403c7a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:09:27.407 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2023-10-05T21:09:27.413 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls 2023-10-05T21:09:27.413 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep test1 2023-10-05T21:09:27.413 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:09:27.413 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '^0$' 2023-10-05T21:09:27.440 INFO:tasks.workunit.client.0.smithi167.stdout:0 2023-10-05T21:09:27.441 INFO:tasks.workunit.client.0.smithi167.stderr:+ RBD_CREATE_ARGS='--image-format 2' 2023-10-05T21:09:27.441 INFO:tasks.workunit.client.0.smithi167.stderr:+ test_others 2023-10-05T21:09:27.441 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'testing import, export, resize, and snapshots...' 2023-10-05T21:09:27.441 INFO:tasks.workunit.client.0.smithi167.stdout:testing import, export, resize, and snapshots... 2023-10-05T21:09:27.442 INFO:tasks.workunit.client.0.smithi167.stderr:+ TMP_FILES='/tmp/img1 /tmp/img1.new /tmp/img2 /tmp/img2.new /tmp/img3 /tmp/img3.new /tmp/img-diff1.new /tmp/img-diff2.new /tmp/img-diff3.new /tmp/img1.snap1 /tmp/img1.snap1 /tmp/img-diff1.snap1' 2023-10-05T21:09:27.442 INFO:tasks.workunit.client.0.smithi167.stderr:+ remove_images 2023-10-05T21:09:27.442 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:27.521 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:27.600 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:27.678 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:27.756 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:27.834 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:27.910 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:27.986 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:28.064 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:28.141 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:28.219 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:28.298 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:28.376 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:28.456 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:28.535 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:28.611 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:28.690 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:28.767 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:29.046 INFO:tasks.workunit.client.0.smithi167.stderr:+ rm -f /tmp/img1 /tmp/img1.new /tmp/img2 /tmp/img2.new /tmp/img3 /tmp/img3.new /tmp/img-diff1.new /tmp/img-diff2.new /tmp/img-diff3.new /tmp/img1.snap1 /tmp/img1.snap1 /tmp/img-diff1.snap1 2023-10-05T21:09:29.046 INFO:tasks.workunit.client.0.smithi167.stderr:+ dd if=/bin/sh of=/tmp/img1 bs=1k count=1 seek=10 2023-10-05T21:09:29.047 INFO:tasks.workunit.client.0.smithi167.stderr:1+0 records in 2023-10-05T21:09:29.047 INFO:tasks.workunit.client.0.smithi167.stderr:1+0 records out 2023-10-05T21:09:29.047 INFO:tasks.workunit.client.0.smithi167.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 3.9592e-05 s, 25.9 MB/s 2023-10-05T21:09:29.047 INFO:tasks.workunit.client.0.smithi167.stderr:+ dd if=/bin/dd of=/tmp/img1 bs=1k count=10 seek=100 2023-10-05T21:09:29.048 INFO:tasks.workunit.client.0.smithi167.stderr:10+0 records in 2023-10-05T21:09:29.048 INFO:tasks.workunit.client.0.smithi167.stderr:10+0 records out 2023-10-05T21:09:29.048 INFO:tasks.workunit.client.0.smithi167.stderr:10240 bytes (10 kB, 10 KiB) copied, 4.9828e-05 s, 206 MB/s 2023-10-05T21:09:29.048 INFO:tasks.workunit.client.0.smithi167.stderr:+ dd if=/bin/rm of=/tmp/img1 bs=1k count=100 seek=1000 2023-10-05T21:09:29.049 INFO:tasks.workunit.client.0.smithi167.stderr:70+1 records in 2023-10-05T21:09:29.049 INFO:tasks.workunit.client.0.smithi167.stderr:70+1 records out 2023-10-05T21:09:29.049 INFO:tasks.workunit.client.0.smithi167.stderr:72208 bytes (72 kB, 71 KiB) copied, 0.00017995 s, 401 MB/s 2023-10-05T21:09:29.049 INFO:tasks.workunit.client.0.smithi167.stderr:+ dd if=/bin/ls of=/tmp/img1 bs=1k seek=10000 2023-10-05T21:09:29.050 INFO:tasks.workunit.client.0.smithi167.stderr:140+1 records in 2023-10-05T21:09:29.050 INFO:tasks.workunit.client.0.smithi167.stderr:140+1 records out 2023-10-05T21:09:29.050 INFO:tasks.workunit.client.0.smithi167.stderr:143376 bytes (143 kB, 140 KiB) copied, 0.000322015 s, 445 MB/s 2023-10-05T21:09:29.050 INFO:tasks.workunit.client.0.smithi167.stderr:+ dd if=/bin/ln of=/tmp/img1 bs=1k seek=100000 2023-10-05T21:09:29.051 INFO:tasks.workunit.client.0.smithi167.stderr:70+1 records in 2023-10-05T21:09:29.051 INFO:tasks.workunit.client.0.smithi167.stderr:70+1 records out 2023-10-05T21:09:29.051 INFO:tasks.workunit.client.0.smithi167.stderr:72312 bytes (72 kB, 71 KiB) copied, 0.000190203 s, 380 MB/s 2023-10-05T21:09:29.051 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd import --image-format 2 /tmp/img1 testimg1 2023-10-05T21:09:29.183 INFO:tasks.workunit.client.0.smithi167.stderr: Importing image: 4% complete... Importing image: 8% complete... Importing image: 12% complete... Importing image: 16% complete... Importing image: 20% complete... Importing image: 24% complete... Importing image: 28% complete... Importing image: 32% complete... Importing image: 36% complete... Importing image: 40% complete... Importing image: 45% complete... Importing image: 49% complete... Importing image: 53% complete... Importing image: 57% complete... Importing image: 61% complete... Importing image: 65% complete... Importing image: 69% complete... Importing image: 73% complete... Importing image: 77% complete... Importing image: 81% complete... Importing image: 85% complete... Importing image: 90% complete... Importing image: 94% complete... Importing image: 98% complete... Importing image: 100% complete...done. 2023-10-05T21:09:29.186 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd resize testimg1 --size=256 --allow-shrink 2023-10-05T21:09:29.227 INFO:tasks.workunit.client.0.smithi167.stderr: Resizing image: 100% complete...done. 2023-10-05T21:09:29.236 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd export testimg1 /tmp/img2 2023-10-05T21:09:29.460 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:09:29.460+0000 7f36946af700 0 --2- 172.21.15.167:0/985238413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559edf8407b0 0x559edf8def80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2023-10-05T21:09:29.549 INFO:tasks.workunit.client.0.smithi167.stderr: Exporting image: 1% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 12% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 23% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 37% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 48% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 62% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 73% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 87% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 98% complete... Exporting image: 100% complete...done. 2023-10-05T21:09:29.552 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap create testimg1 --snap=snap1 2023-10-05T21:09:29.757 INFO:tasks.workunit.client.0.smithi167.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-05T21:09:29.764 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd resize testimg1 --size=128 2023-10-05T21:09:29.799 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: shrinking an image is only allowed with the --allow-shrink flag 2023-10-05T21:09:29.803 INFO:tasks.workunit.client.0.smithi167.stderr:+ true 2023-10-05T21:09:29.803 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd resize testimg1 --size=128 --allow-shrink 2023-10-05T21:09:29.846 INFO:tasks.workunit.client.0.smithi167.stderr: Resizing image: 50% complete... Resizing image: 51% complete... Resizing image: 53% complete... Resizing image: 54% complete... Resizing image: 56% complete... Resizing image: 57% complete... Resizing image: 59% complete... Resizing image: 60% complete... Resizing image: 62% complete... Resizing image: 64% complete... Resizing image: 65% complete... Resizing image: 67% complete... Resizing image: 68% complete... Resizing image: 70% complete... Resizing image: 71% complete... Resizing image: 73% complete... Resizing image: 75% complete... Resizing image: 76% complete... Resizing image: 78% complete... Resizing image: 79% complete... Resizing image: 81% complete... Resizing image: 82% complete... Resizing image: 84% complete... Resizing image: 85% complete... Resizing image: 87% complete... Resizing image: 89% complete... Resizing image: 90% complete... Resizing image: 92% complete... Resizing image: 93% complete... Resizing image: 95% complete... Resizing image: 96% complete... Resizing image: 98% complete... Resizing image: 100% complete...done. 2023-10-05T21:09:29.855 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd export testimg1 /tmp/img3 2023-10-05T21:09:29.926 INFO:tasks.workunit.client.0.smithi167.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2023-10-05T21:09:29.930 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info testimg1 2023-10-05T21:09:29.930 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'size 128 MiB' 2023-10-05T21:09:29.965 INFO:tasks.workunit.client.0.smithi167.stdout: size 128 MiB in 32 objects 2023-10-05T21:09:29.966 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info --snap=snap1 testimg1 2023-10-05T21:09:29.966 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'size 256 MiB' 2023-10-05T21:09:30.002 INFO:tasks.workunit.client.0.smithi167.stdout: size 256 MiB in 64 objects 2023-10-05T21:09:30.003 INFO:tasks.workunit.client.0.smithi167.stderr:+ rm -rf /tmp/diff-testimg1-1 /tmp/diff-testimg1-2 2023-10-05T21:09:30.004 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd export-diff testimg1 --snap=snap1 /tmp/diff-testimg1-1 2023-10-05T21:09:30.058 INFO:tasks.workunit.client.0.smithi167.stderr: Exporting image: 3% complete... Exporting image: 37% complete... Exporting image: 100% complete...done. 2023-10-05T21:09:30.061 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd export-diff testimg1 --from-snap=snap1 /tmp/diff-testimg1-2 2023-10-05T21:09:30.103 INFO:tasks.workunit.client.0.smithi167.stderr: Exporting image: 100% complete...done. 2023-10-05T21:09:30.106 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 2 --size=1 testimg-diff1 2023-10-05T21:09:30.150 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd import-diff --sparse-size 8K /tmp/diff-testimg1-1 testimg-diff1 2023-10-05T21:09:30.774 INFO:tasks.workunit.client.0.smithi167.stderr: Importing image diff: 22% complete... Importing image diff: 63% complete... Importing image diff: 99% complete... Importing image diff: 100% complete...done. 2023-10-05T21:09:30.784 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd import-diff --sparse-size 8K /tmp/diff-testimg1-2 testimg-diff1 2023-10-05T21:09:30.834 INFO:tasks.workunit.client.0.smithi167.stderr: Importing image diff: 68% complete... Importing image diff: 96% complete... Importing image diff: 100% complete...done. 2023-10-05T21:09:30.846 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info testimg1 2023-10-05T21:09:30.846 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'size 128 MiB' 2023-10-05T21:09:30.881 INFO:tasks.workunit.client.0.smithi167.stdout: size 128 MiB in 32 objects 2023-10-05T21:09:30.882 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info --snap=snap1 testimg1 2023-10-05T21:09:30.882 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'size 256 MiB' 2023-10-05T21:09:30.920 INFO:tasks.workunit.client.0.smithi167.stdout: size 256 MiB in 64 objects 2023-10-05T21:09:30.920 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info testimg-diff1 2023-10-05T21:09:30.920 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'size 128 MiB' 2023-10-05T21:09:30.955 INFO:tasks.workunit.client.0.smithi167.stdout: size 128 MiB in 32 objects 2023-10-05T21:09:30.955 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info --snap=snap1 testimg-diff1 2023-10-05T21:09:30.955 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'size 256 MiB' 2023-10-05T21:09:30.992 INFO:tasks.workunit.client.0.smithi167.stdout: size 256 MiB in 64 objects 2023-10-05T21:09:30.992 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd copy testimg1 --snap=snap1 testimg2 2023-10-05T21:09:31.125 INFO:tasks.workunit.client.0.smithi167.stderr: Image copy: 3% complete... Image copy: 37% complete... Image copy: 100% complete... Image copy: 100% complete...done. 2023-10-05T21:09:31.128 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd copy testimg1 testimg3 2023-10-05T21:09:31.270 INFO:tasks.workunit.client.0.smithi167.stderr: Image copy: 3% complete... Image copy: 6% complete... Image copy: 9% complete... Image copy: 12% complete... Image copy: 15% complete... Image copy: 18% complete... Image copy: 21% complete... Image copy: 25% complete... Image copy: 28% complete... Image copy: 31% complete... Image copy: 34% complete... Image copy: 37% complete... Image copy: 40% complete... Image copy: 43% complete... Image copy: 46% complete... Image copy: 50% complete... Image copy: 53% complete... Image copy: 56% complete... Image copy: 59% complete... Image copy: 62% complete... Image copy: 65% complete... Image copy: 68% complete... Image copy: 71% complete... Image copy: 75% complete... Image copy: 78% complete... Image copy: 81% complete... Image copy: 84% complete... Image copy: 87% complete... Image copy: 90% complete... Image copy: 93% complete... Image copy: 96% complete... Image copy: 100% complete... Image copy: 100% complete...done. 2023-10-05T21:09:31.273 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd copy testimg-diff1 --sparse-size 768K --snap=snap1 testimg-diff2 2023-10-05T21:09:31.377 INFO:tasks.workunit.client.0.smithi167.stderr: Image copy: 3% complete... Image copy: 37% complete... Image copy: 100% complete... Image copy: 100% complete...done. 2023-10-05T21:09:31.381 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd copy testimg-diff1 --sparse-size 768K testimg-diff3 2023-10-05T21:09:31.496 INFO:tasks.workunit.client.0.smithi167.stderr: Image copy: 3% complete... Image copy: 6% complete... Image copy: 9% complete... Image copy: 12% complete... Image copy: 15% complete... Image copy: 18% complete... Image copy: 21% complete... Image copy: 25% complete... Image copy: 28% complete... Image copy: 31% complete... Image copy: 34% complete... Image copy: 37% complete... Image copy: 40% complete... Image copy: 43% complete... Image copy: 46% complete... Image copy: 50% complete... Image copy: 53% complete... Image copy: 56% complete... Image copy: 59% complete... Image copy: 62% complete... Image copy: 65% complete... Image copy: 68% complete... Image copy: 71% complete... Image copy: 75% complete... Image copy: 78% complete... Image copy: 81% complete... Image copy: 84% complete... Image copy: 87% complete... Image copy: 90% complete... Image copy: 93% complete... Image copy: 96% complete... Image copy: 100% complete... Image copy: 100% complete...done. 2023-10-05T21:09:31.500 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info testimg2 2023-10-05T21:09:31.500 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'size 256 MiB' 2023-10-05T21:09:31.533 INFO:tasks.workunit.client.0.smithi167.stdout: size 256 MiB in 64 objects 2023-10-05T21:09:31.534 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info testimg3 2023-10-05T21:09:31.534 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'size 128 MiB' 2023-10-05T21:09:31.567 INFO:tasks.workunit.client.0.smithi167.stdout: size 128 MiB in 32 objects 2023-10-05T21:09:31.567 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info testimg-diff2 2023-10-05T21:09:31.568 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'size 256 MiB' 2023-10-05T21:09:31.601 INFO:tasks.workunit.client.0.smithi167.stdout: size 256 MiB in 64 objects 2023-10-05T21:09:31.602 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info testimg-diff3 2023-10-05T21:09:31.602 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'size 128 MiB' 2023-10-05T21:09:31.634 INFO:tasks.workunit.client.0.smithi167.stdout: size 128 MiB in 32 objects 2023-10-05T21:09:31.635 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd deep copy testimg1 testimg4 2023-10-05T21:09:32.660 INFO:tasks.workunit.client.0.smithi167.stderr: Image deep copy: 1% complete... Image deep copy: 3% complete... Image deep copy: 4% complete... Image deep copy: 6% complete... Image deep copy: 7% complete... Image deep copy: 9% complete... Image deep copy: 10% complete... Image deep copy: 12% complete... Image deep copy: 14% complete... Image deep copy: 15% complete... Image deep copy: 17% complete... Image deep copy: 18% complete... Image deep copy: 20% complete... Image deep copy: 21% complete... Image deep copy: 23% complete... Image deep copy: 25% complete... Image deep copy: 26% complete... Image deep copy: 28% complete... Image deep copy: 29% complete... Image deep copy: 31% complete... Image deep copy: 32% complete... Image deep copy: 34% complete... Image deep copy: 35% complete... Image deep copy: 37% complete... Image deep copy: 39% complete... Image deep copy: 40% complete... Image deep copy: 42% complete... Image deep copy: 43% complete... Image deep copy: 45% complete... Image deep copy: 46% complete... Image deep copy: 48% complete... Image deep copy: 50% complete... Image deep copy: 51% complete... Image deep copy: 53% complete... Image deep copy: 54% complete... Image deep copy: 56% complete... Image deep copy: 57% complete... Image deep copy: 59% complete... Image deep copy: 60% complete... Image deep copy: 62% complete... Image deep copy: 64% complete... Image deep copy: 65% complete... Image deep copy: 67% complete... Image deep copy: 68% complete... Image deep copy: 70% complete... Image deep copy: 71% complete... Image deep copy: 73% complete... Image deep copy: 75% complete... Image deep copy: 76% complete... Image deep copy: 78% complete... Image deep copy: 79% complete... Image deep copy: 81% complete... Image deep copy: 82% complete... Image deep copy: 84% complete... Image deep copy: 85% complete... Image deep copy: 87% complete... Image deep copy: 89% complete... Image deep copy: 90% complete... Image deep copy: 92% complete... Image deep copy: 93% complete... Image deep copy: 95% complete... Image deep copy: 96% complete... Image deep copy: 98% complete... Image deep copy: 100% complete... Image deep copy: 100% complete...done. 2023-10-05T21:09:32.664 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd deep copy testimg1 --snap=snap1 testimg5 2023-10-05T21:09:33.672 INFO:tasks.workunit.client.0.smithi167.stderr: Image deep copy: 1% complete... Image deep copy: 3% complete... Image deep copy: 4% complete... Image deep copy: 6% complete... Image deep copy: 7% complete... Image deep copy: 9% complete... Image deep copy: 10% complete... Image deep copy: 12% complete... Image deep copy: 14% complete... Image deep copy: 15% complete... Image deep copy: 17% complete... Image deep copy: 18% complete... Image deep copy: 20% complete... Image deep copy: 21% complete... Image deep copy: 23% complete... Image deep copy: 25% complete... Image deep copy: 26% complete... Image deep copy: 28% complete... Image deep copy: 29% complete... Image deep copy: 31% complete... Image deep copy: 32% complete... Image deep copy: 34% complete... Image deep copy: 35% complete... Image deep copy: 37% complete... Image deep copy: 39% complete... Image deep copy: 40% complete... Image deep copy: 42% complete... Image deep copy: 43% complete... Image deep copy: 45% complete... Image deep copy: 46% complete... Image deep copy: 48% complete... Image deep copy: 50% complete... Image deep copy: 51% complete... Image deep copy: 53% complete... Image deep copy: 54% complete... Image deep copy: 56% complete... Image deep copy: 57% complete... Image deep copy: 59% complete... Image deep copy: 60% complete... Image deep copy: 62% complete... Image deep copy: 64% complete... Image deep copy: 65% complete... Image deep copy: 67% complete... Image deep copy: 68% complete... Image deep copy: 70% complete... Image deep copy: 71% complete... Image deep copy: 73% complete... Image deep copy: 75% complete... Image deep copy: 76% complete... Image deep copy: 78% complete... Image deep copy: 79% complete... Image deep copy: 81% complete... Image deep copy: 82% complete... Image deep copy: 84% complete... Image deep copy: 85% complete... Image deep copy: 87% complete... Image deep copy: 89% complete... Image deep copy: 90% complete... Image deep copy: 92% complete... Image deep copy: 93% complete... Image deep copy: 95% complete... Image deep copy: 96% complete... Image deep copy: 98% complete... Image deep copy: 100% complete... Image deep copy: 100% complete...done. 2023-10-05T21:09:33.676 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info testimg4 2023-10-05T21:09:33.676 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'size 128 MiB' 2023-10-05T21:09:33.714 INFO:tasks.workunit.client.0.smithi167.stdout: size 128 MiB in 32 objects 2023-10-05T21:09:33.714 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info testimg5 2023-10-05T21:09:33.714 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'size 256 MiB' 2023-10-05T21:09:33.755 INFO:tasks.workunit.client.0.smithi167.stdout: size 256 MiB in 64 objects 2023-10-05T21:09:33.756 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap ls testimg4 2023-10-05T21:09:33.756 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep -v SNAPID 2023-10-05T21:09:33.756 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:09:33.756 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 1 2023-10-05T21:09:33.790 INFO:tasks.workunit.client.0.smithi167.stdout:1 2023-10-05T21:09:33.790 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap ls testimg4 2023-10-05T21:09:33.790 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '.*snap1.*' 2023-10-05T21:09:33.824 INFO:tasks.workunit.client.0.smithi167.stdout: 26 snap1 256 MiB Thu Oct 5 21:09:32 2023 2023-10-05T21:09:33.824 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd export testimg1 /tmp/img1.new 2023-10-05T21:09:33.898 INFO:tasks.workunit.client.0.smithi167.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2023-10-05T21:09:33.901 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd export testimg2 /tmp/img2.new 2023-10-05T21:09:34.003 INFO:tasks.workunit.client.0.smithi167.stderr: Exporting image: 1% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 12% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 23% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 37% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 48% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 62% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 73% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 87% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 98% complete... Exporting image: 100% complete...done. 2023-10-05T21:09:34.006 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd export testimg3 /tmp/img3.new 2023-10-05T21:09:34.079 INFO:tasks.workunit.client.0.smithi167.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2023-10-05T21:09:34.083 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd export testimg-diff1 /tmp/img-diff1.new 2023-10-05T21:09:34.156 INFO:tasks.workunit.client.0.smithi167.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2023-10-05T21:09:34.160 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd export testimg-diff2 /tmp/img-diff2.new 2023-10-05T21:09:34.268 INFO:tasks.workunit.client.0.smithi167.stderr: Exporting image: 1% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 12% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 23% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 37% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 48% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 62% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 73% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 87% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 98% complete... Exporting image: 100% complete...done. 2023-10-05T21:09:34.271 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd export testimg-diff3 /tmp/img-diff3.new 2023-10-05T21:09:34.343 INFO:tasks.workunit.client.0.smithi167.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2023-10-05T21:09:34.346 INFO:tasks.workunit.client.0.smithi167.stderr:+ cmp /tmp/img2 /tmp/img2.new 2023-10-05T21:09:34.542 INFO:tasks.workunit.client.0.smithi167.stderr:+ cmp /tmp/img3 /tmp/img3.new 2023-10-05T21:09:34.640 INFO:tasks.workunit.client.0.smithi167.stderr:+ cmp /tmp/img2 /tmp/img-diff2.new 2023-10-05T21:09:34.801 INFO:tasks.workunit.client.0.smithi167.stderr:+ cmp /tmp/img3 /tmp/img-diff3.new 2023-10-05T21:09:34.882 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap rollback --snap=snap1 testimg1 2023-10-05T21:09:34.989 INFO:tasks.workunit.client.0.smithi167.stderr: Rolling back to snapshot: 1% complete... Rolling back to snapshot: 3% complete... Rolling back to snapshot: 4% complete... Rolling back to snapshot: 6% complete... Rolling back to snapshot: 7% complete... Rolling back to snapshot: 9% complete... Rolling back to snapshot: 10% complete... Rolling back to snapshot: 12% complete... Rolling back to snapshot: 14% complete... Rolling back to snapshot: 15% complete... Rolling back to snapshot: 17% complete... Rolling back to snapshot: 18% complete... Rolling back to snapshot: 20% complete... Rolling back to snapshot: 21% complete... Rolling back to snapshot: 23% complete... Rolling back to snapshot: 25% complete... Rolling back to snapshot: 26% complete... Rolling back to snapshot: 28% complete... Rolling back to snapshot: 29% complete... Rolling back to snapshot: 31% complete... Rolling back to snapshot: 32% complete... Rolling back to snapshot: 34% complete... Rolling back to snapshot: 35% complete... Rolling back to snapshot: 37% complete... Rolling back to snapshot: 39% complete... Rolling back to snapshot: 40% complete... Rolling back to snapshot: 42% complete... Rolling back to snapshot: 43% complete... Rolling back to snapshot: 45% complete... Rolling back to snapshot: 46% complete... Rolling back to snapshot: 48% complete... Rolling back to snapshot: 50% complete... Rolling back to snapshot: 51% complete... Rolling back to snapshot: 53% complete... Rolling back to snapshot: 54% complete... Rolling back to snapshot: 56% complete... Rolling back to snapshot: 57% complete... Rolling back to snapshot: 59% complete... Rolling back to snapshot: 60% complete... Rolling back to snapshot: 62% complete... Rolling back to snapshot: 64% complete... Rolling back to snapshot: 65% complete... Rolling back to snapshot: 67% complete... Rolling back to snapshot: 68% complete... Rolling back to snapshot: 70% complete... Rolling back to snapshot: 71% complete... Rolling back to snapshot: 73% complete... Rolling back to snapshot: 75% complete... Rolling back to snapshot: 76% complete... Rolling back to snapshot: 78% complete... Rolling back to snapshot: 79% complete... Rolling back to snapshot: 81% complete... Rolling back to snapshot: 82% complete... Rolling back to snapshot: 84% complete... Rolling back to snapshot: 85% complete... Rolling back to snapshot: 87% complete... Rolling back to snapshot: 89% complete... Rolling back to snapshot: 90% complete... Rolling back to snapshot: 92% complete... Rolling back to snapshot: 93% complete... Rolling back to snapshot: 95% complete... Rolling back to snapshot: 96% complete... Rolling back to snapshot: 98% complete... Rolling back to snapshot: 100% complete...done. 2023-10-05T21:09:34.999 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap rollback --snap=snap1 testimg-diff1 2023-10-05T21:09:35.111 INFO:tasks.workunit.client.0.smithi167.stderr: Rolling back to snapshot: 1% complete... Rolling back to snapshot: 3% complete... Rolling back to snapshot: 4% complete... Rolling back to snapshot: 6% complete... Rolling back to snapshot: 7% complete... Rolling back to snapshot: 9% complete... Rolling back to snapshot: 10% complete... Rolling back to snapshot: 12% complete... Rolling back to snapshot: 14% complete... Rolling back to snapshot: 15% complete... Rolling back to snapshot: 17% complete... Rolling back to snapshot: 18% complete... Rolling back to snapshot: 20% complete... Rolling back to snapshot: 21% complete... Rolling back to snapshot: 23% complete... Rolling back to snapshot: 25% complete... Rolling back to snapshot: 26% complete... Rolling back to snapshot: 28% complete... Rolling back to snapshot: 29% complete... Rolling back to snapshot: 31% complete... Rolling back to snapshot: 32% complete... Rolling back to snapshot: 34% complete... Rolling back to snapshot: 35% complete... Rolling back to snapshot: 37% complete... Rolling back to snapshot: 39% complete... Rolling back to snapshot: 40% complete... Rolling back to snapshot: 42% complete... Rolling back to snapshot: 43% complete... Rolling back to snapshot: 45% complete... Rolling back to snapshot: 46% complete... Rolling back to snapshot: 48% complete... Rolling back to snapshot: 50% complete... Rolling back to snapshot: 51% complete... Rolling back to snapshot: 53% complete... Rolling back to snapshot: 54% complete... Rolling back to snapshot: 56% complete... Rolling back to snapshot: 57% complete... Rolling back to snapshot: 59% complete... Rolling back to snapshot: 60% complete... Rolling back to snapshot: 62% complete... Rolling back to snapshot: 64% complete... Rolling back to snapshot: 65% complete... Rolling back to snapshot: 67% complete... Rolling back to snapshot: 68% complete... Rolling back to snapshot: 70% complete... Rolling back to snapshot: 71% complete... Rolling back to snapshot: 73% complete... Rolling back to snapshot: 75% complete... Rolling back to snapshot: 76% complete... Rolling back to snapshot: 78% complete... Rolling back to snapshot: 79% complete... Rolling back to snapshot: 81% complete... Rolling back to snapshot: 82% complete... Rolling back to snapshot: 84% complete... Rolling back to snapshot: 85% complete... Rolling back to snapshot: 87% complete... Rolling back to snapshot: 89% complete... Rolling back to snapshot: 90% complete... Rolling back to snapshot: 92% complete... Rolling back to snapshot: 93% complete... Rolling back to snapshot: 95% complete... Rolling back to snapshot: 96% complete... Rolling back to snapshot: 98% complete... Rolling back to snapshot: 100% complete...done. 2023-10-05T21:09:35.125 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info testimg1 2023-10-05T21:09:35.125 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'size 256 MiB' 2023-10-05T21:09:35.165 INFO:tasks.workunit.client.0.smithi167.stdout: size 256 MiB in 64 objects 2023-10-05T21:09:35.165 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info testimg-diff1 2023-10-05T21:09:35.165 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'size 256 MiB' 2023-10-05T21:09:35.206 INFO:tasks.workunit.client.0.smithi167.stdout: size 256 MiB in 64 objects 2023-10-05T21:09:35.206 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd export testimg1 /tmp/img1.snap1 2023-10-05T21:09:35.312 INFO:tasks.workunit.client.0.smithi167.stderr: Exporting image: 1% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 12% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 23% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 37% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 48% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 62% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 73% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 87% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 98% complete... Exporting image: 100% complete...done. 2023-10-05T21:09:35.315 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd export testimg-diff1 /tmp/img-diff1.snap1 2023-10-05T21:09:35.416 INFO:tasks.workunit.client.0.smithi167.stderr: Exporting image: 1% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 12% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 23% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 37% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 48% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 62% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 73% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 87% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 98% complete... Exporting image: 100% complete...done. 2023-10-05T21:09:35.419 INFO:tasks.workunit.client.0.smithi167.stderr:+ cmp /tmp/img2 /tmp/img1.snap1 2023-10-05T21:09:35.570 INFO:tasks.workunit.client.0.smithi167.stderr:+ cmp /tmp/img2 /tmp/img-diff1.snap1 2023-10-05T21:09:35.720 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm testimg2 2023-10-05T21:09:35.834 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 1% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 12% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 23% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 37% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 48% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 62% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 73% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 87% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 98% complete...2023-10-05T21:09:35.833+0000 7faf9ef90700 0 -- 172.21.15.167:0/345081888 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x561545c32870 msgr2=0x561545cd7e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:09:35.843 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:09:35.846 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm testimg3 2023-10-05T21:09:35.941 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete...2023-10-05T21:09:35.940+0000 7f36bc9ce700 0 -- 172.21.15.167:0/1057046518 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x7f36a003a3d0 msgr2=0x7f36a003c7d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:09:35.956 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:09:35.959 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create testimg2 -s 0 2023-10-05T21:09:36.183 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:09:36.182+0000 7f17a6e67700 0 --2- 172.21.15.167:0/4054736154 >> v2:172.21.15.167:3300/0 conn(0x55eecb7a6a60 0x55eecb7a6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2023-10-05T21:09:36.389 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:09:36.388+0000 7f17af376540 -1 librbd: Forced V1 image creation. 2023-10-05T21:09:36.397 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd cp testimg2 testimg3 2023-10-05T21:09:36.450 INFO:tasks.workunit.client.0.smithi167.stderr: Image copy: 100% complete...done. 2023-10-05T21:09:36.454 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd deep cp testimg2 testimg6 2023-10-05T21:09:36.508 INFO:tasks.workunit.client.0.smithi167.stderr: Image deep copy: 100% complete...done. 2023-10-05T21:09:36.511 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap rm --snap=snap1 testimg1 2023-10-05T21:09:36.611 INFO:tasks.workunit.client.0.smithi167.stderr: Removing snap: 100% complete...done. 2023-10-05T21:09:36.618 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap rm --snap=snap1 testimg-diff1 2023-10-05T21:09:37.618 INFO:tasks.workunit.client.0.smithi167.stderr: Removing snap: 100% complete...done. 2023-10-05T21:09:37.630 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info --snap=snap1 testimg1 2023-10-05T21:09:37.630 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'error setting snapshot context: (2) No such file or directory' 2023-10-05T21:09:37.669 INFO:tasks.workunit.client.0.smithi167.stdout:error setting snapshot context: (2) No such file or directory 2023-10-05T21:09:37.669 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info --snap=snap1 testimg-diff1 2023-10-05T21:09:37.669 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'error setting snapshot context: (2) No such file or directory' 2023-10-05T21:09:37.705 INFO:tasks.workunit.client.0.smithi167.stdout:error setting snapshot context: (2) No such file or directory 2023-10-05T21:09:37.705 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd sparsify testimg1 2023-10-05T21:09:37.774 INFO:tasks.workunit.client.0.smithi167.stderr: Image sparsify: 1% complete... Image sparsify: 3% complete... Image sparsify: 4% complete... Image sparsify: 6% complete... Image sparsify: 7% complete... Image sparsify: 9% complete... Image sparsify: 10% complete... Image sparsify: 12% complete... Image sparsify: 14% complete... Image sparsify: 15% complete... Image sparsify: 17% complete... Image sparsify: 18% complete... Image sparsify: 20% complete... Image sparsify: 21% complete... Image sparsify: 23% complete... Image sparsify: 25% complete... Image sparsify: 26% complete... Image sparsify: 28% complete... Image sparsify: 29% complete... Image sparsify: 31% complete... Image sparsify: 32% complete... Image sparsify: 34% complete... Image sparsify: 35% complete... Image sparsify: 37% complete... Image sparsify: 39% complete... Image sparsify: 40% complete... Image sparsify: 42% complete... Image sparsify: 43% complete... Image sparsify: 45% complete... Image sparsify: 46% complete... Image sparsify: 48% complete... Image sparsify: 50% complete... Image sparsify: 51% complete... Image sparsify: 53% complete... Image sparsify: 54% complete... Image sparsify: 56% complete... Image sparsify: 57% complete... Image sparsify: 59% complete... Image sparsify: 60% complete... Image sparsify: 62% complete... Image sparsify: 64% complete... Image sparsify: 65% complete... Image sparsify: 67% complete... Image sparsify: 68% complete... Image sparsify: 70% complete... Image sparsify: 71% complete... Image sparsify: 73% complete... Image sparsify: 75% complete... Image sparsify: 76% complete... Image sparsify: 78% complete... Image sparsify: 79% complete... Image sparsify: 81% complete... Image sparsify: 82% complete... Image sparsify: 84% complete... Image sparsify: 85% complete... Image sparsify: 87% complete... Image sparsify: 89% complete... Image sparsify: 90% complete... Image sparsify: 92% complete... Image sparsify: 93% complete... Image sparsify: 95% complete... Image sparsify: 96% complete... Image sparsify: 98% complete... Image sparsify: 100% complete...done. 2023-10-05T21:09:37.931 INFO:tasks.workunit.client.0.smithi167.stderr:+ remove_images 2023-10-05T21:09:37.931 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:37.931 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:38.203 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:38.318 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:38.775 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:39.754 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:39.880 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:40.056 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:40.221 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:40.379 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:40.463 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:40.541 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:40.618 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:40.700 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:40.784 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:40.865 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:40.945 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:41.025 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:41.108 INFO:tasks.workunit.client.0.smithi167.stderr:+ rm -f /tmp/img1 /tmp/img1.new /tmp/img2 /tmp/img2.new /tmp/img3 /tmp/img3.new /tmp/img-diff1.new /tmp/img-diff2.new /tmp/img-diff3.new /tmp/img1.snap1 /tmp/img1.snap1 /tmp/img-diff1.snap1 2023-10-05T21:09:41.218 INFO:tasks.workunit.client.0.smithi167.stderr:+ test_locking 2023-10-05T21:09:41.218 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'testing locking...' 2023-10-05T21:09:41.219 INFO:tasks.workunit.client.0.smithi167.stdout:testing locking... 2023-10-05T21:09:41.219 INFO:tasks.workunit.client.0.smithi167.stderr:+ remove_images 2023-10-05T21:09:41.220 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:41.299 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:41.579 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:41.662 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:41.743 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:41.822 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:41.900 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:41.978 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:42.058 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:42.136 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:42.215 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:42.294 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:42.375 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:42.452 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:42.530 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:42.608 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:42.688 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:42.766 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:42.845 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 2 -s 1 test1 2023-10-05T21:09:42.890 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd lock list test1 2023-10-05T21:09:42.890 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:09:42.890 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '^0$' 2023-10-05T21:09:42.924 INFO:tasks.workunit.client.0.smithi167.stdout:0 2023-10-05T21:09:42.925 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd lock add test1 id 2023-10-05T21:09:42.971 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd lock list test1 2023-10-05T21:09:42.971 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep ' 1 ' 2023-10-05T21:09:43.006 INFO:tasks.workunit.client.0.smithi167.stdout:There is 1 exclusive lock on this image. 2023-10-05T21:09:43.006 INFO:tasks.workunit.client.0.smithi167.stderr:++ rbd lock list test1 2023-10-05T21:09:43.006 INFO:tasks.workunit.client.0.smithi167.stderr:++ tail -n 1 2023-10-05T21:09:43.007 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{print $1;}' 2023-10-05T21:09:43.041 INFO:tasks.workunit.client.0.smithi167.stderr:+ LOCKER=client.8456 2023-10-05T21:09:43.041 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd lock remove test1 id client.8456 2023-10-05T21:09:43.639 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd lock list test1 2023-10-05T21:09:43.639 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:09:43.639 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '^0$' 2023-10-05T21:09:43.676 INFO:tasks.workunit.client.0.smithi167.stdout:0 2023-10-05T21:09:43.676 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd lock add test1 id --shared tag 2023-10-05T21:09:43.726 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd lock list test1 2023-10-05T21:09:43.727 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep ' 1 ' 2023-10-05T21:09:43.761 INFO:tasks.workunit.client.0.smithi167.stdout:There is 1 shared lock on this image. 2023-10-05T21:09:43.761 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd lock add test1 id --shared tag 2023-10-05T21:09:43.806 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd lock list test1 2023-10-05T21:09:43.806 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep ' 2 ' 2023-10-05T21:09:43.841 INFO:tasks.workunit.client.0.smithi167.stdout:There are 2 shared locks on this image. 2023-10-05T21:09:43.841 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd lock add test1 id2 --shared tag 2023-10-05T21:09:43.886 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd lock list test1 2023-10-05T21:09:43.886 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep ' 3 ' 2023-10-05T21:09:43.920 INFO:tasks.workunit.client.0.smithi167.stdout:There are 3 shared locks on this image. 2023-10-05T21:09:43.921 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd lock list test1 2023-10-05T21:09:43.921 INFO:tasks.workunit.client.0.smithi167.stderr:+ tail -n 1 2023-10-05T21:09:43.921 INFO:tasks.workunit.client.0.smithi167.stderr:+ awk '{print $2, $1;}' 2023-10-05T21:09:43.921 INFO:tasks.workunit.client.0.smithi167.stderr:+ xargs rbd lock remove test1 2023-10-05T21:09:44.643 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info test1 2023-10-05T21:09:44.643 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep -qE 'features:.*exclusive' 2023-10-05T21:09:44.679 INFO:tasks.workunit.client.0.smithi167.stderr:++ rbd lock list test1 2023-10-05T21:09:44.718 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -n 'There are 2 shared locks on this image. 2023-10-05T21:09:44.718 INFO:tasks.workunit.client.0.smithi167.stderr:Lock tag: tag 2023-10-05T21:09:44.718 INFO:tasks.workunit.client.0.smithi167.stderr:Locker ID Address 2023-10-05T21:09:44.718 INFO:tasks.workunit.client.0.smithi167.stderr:client.8471 id 172.21.15.167:0/3876096948 2023-10-05T21:09:44.718 INFO:tasks.workunit.client.0.smithi167.stderr:client.8477 id 172.21.15.167:0/3901649400' ']' 2023-10-05T21:09:44.718 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd lock list test1 2023-10-05T21:09:44.719 INFO:tasks.workunit.client.0.smithi167.stderr:+ tail -n 1 2023-10-05T21:09:44.719 INFO:tasks.workunit.client.0.smithi167.stderr:+ awk '{print $2, $1;}' 2023-10-05T21:09:44.719 INFO:tasks.workunit.client.0.smithi167.stderr:+ xargs rbd lock remove test1 2023-10-05T21:09:45.646 INFO:tasks.workunit.client.0.smithi167.stderr:++ rbd lock list test1 2023-10-05T21:09:45.688 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -n 'There is 1 shared lock on this image. 2023-10-05T21:09:45.688 INFO:tasks.workunit.client.0.smithi167.stderr:Lock tag: tag 2023-10-05T21:09:45.688 INFO:tasks.workunit.client.0.smithi167.stderr:Locker ID Address 2023-10-05T21:09:45.688 INFO:tasks.workunit.client.0.smithi167.stderr:client.8471 id 172.21.15.167:0/3876096948' ']' 2023-10-05T21:09:45.688 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd lock list test1 2023-10-05T21:09:45.689 INFO:tasks.workunit.client.0.smithi167.stderr:+ tail -n 1 2023-10-05T21:09:45.689 INFO:tasks.workunit.client.0.smithi167.stderr:+ awk '{print $2, $1;}' 2023-10-05T21:09:45.689 INFO:tasks.workunit.client.0.smithi167.stderr:+ xargs rbd lock remove test1 2023-10-05T21:09:46.614 INFO:tasks.workunit.client.0.smithi167.stderr:++ rbd lock list test1 2023-10-05T21:09:46.656 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' -n '' ']' 2023-10-05T21:09:46.656 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test1 2023-10-05T21:09:46.763 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:09:46.761+0000 7f275a163700 0 -- 172.21.15.167:0/3232484265 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f2740006580 msgr2=0x7f2740011ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:09:46.764 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:09:46.767 INFO:tasks.workunit.client.0.smithi167.stderr:+ test_clone 2023-10-05T21:09:46.767 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'testing clone...' 2023-10-05T21:09:46.767 INFO:tasks.workunit.client.0.smithi167.stdout:testing clone... 2023-10-05T21:09:46.768 INFO:tasks.workunit.client.0.smithi167.stderr:+ remove_images 2023-10-05T21:09:46.768 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:46.846 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:47.126 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:47.204 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:47.484 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:47.564 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:47.643 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:47.721 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:47.800 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:47.878 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:47.958 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:48.036 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:48.117 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:48.195 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:48.275 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:48.352 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:48.432 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:48.510 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:48.589 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create test1 --image-format 2 -s 1 2023-10-05T21:09:48.635 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap create test1@s1 2023-10-05T21:09:49.660 INFO:tasks.workunit.client.0.smithi167.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-05T21:09:49.669 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap protect test1@s1 2023-10-05T21:09:49.716 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph osd pool create rbd2 8 2023-10-05T21:09:50.777 INFO:tasks.workunit.client.0.smithi167.stderr:pool 'rbd2' already exists 2023-10-05T21:09:50.786 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd pool init rbd2 2023-10-05T21:09:53.663 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd clone test1@s1 rbd2/clone 2023-10-05T21:09:53.729 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd -p rbd2 ls 2023-10-05T21:09:53.729 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep clone 2023-10-05T21:09:53.759 INFO:tasks.workunit.client.0.smithi167.stdout:clone 2023-10-05T21:09:53.759 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd -p rbd2 ls -l 2023-10-05T21:09:53.759 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep clone 2023-10-05T21:09:53.759 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep test1@s1 2023-10-05T21:09:53.802 INFO:tasks.workunit.client.0.smithi167.stdout:clone 1 MiB rbd/test1@s1 2 2023-10-05T21:09:53.803 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls 2023-10-05T21:09:53.803 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep -v clone 2023-10-05T21:09:53.830 INFO:tasks.workunit.client.0.smithi167.stdout:test1 2023-10-05T21:09:53.830 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd flatten rbd2/clone 2023-10-05T21:09:53.873 INFO:tasks.workunit.client.0.smithi167.stderr: Image flatten: 100% complete...done. 2023-10-05T21:09:53.880 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap create rbd2/clone@s1 2023-10-05T21:09:54.668 INFO:tasks.workunit.client.0.smithi167.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-05T21:09:54.675 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap protect rbd2/clone@s1 2023-10-05T21:09:54.718 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd clone rbd2/clone@s1 clone2 2023-10-05T21:09:54.789 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls 2023-10-05T21:09:54.790 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep clone2 2023-10-05T21:09:54.817 INFO:tasks.workunit.client.0.smithi167.stdout:clone2 2023-10-05T21:09:54.817 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls -l 2023-10-05T21:09:54.817 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep clone2 2023-10-05T21:09:54.818 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep rbd2/clone@s1 2023-10-05T21:09:54.860 INFO:tasks.workunit.client.0.smithi167.stdout:clone2 1 MiB rbd2/clone@s1 2 2023-10-05T21:09:54.860 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd -p rbd2 ls 2023-10-05T21:09:54.860 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep -v clone2 2023-10-05T21:09:54.889 INFO:tasks.workunit.client.0.smithi167.stdout:clone 2023-10-05T21:09:54.889 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm clone2 2023-10-05T21:09:55.005 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:09:55.003+0000 7f0c3c9a3700 0 -- 172.21.15.167:0/5946984 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x55a029aeb870 msgr2=0x55a029b8f850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-05T21:09:55.011 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:09:55.014 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap unprotect rbd2/clone@s1 2023-10-05T21:09:55.055 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap rm rbd2/clone@s1 2023-10-05T21:09:55.664 INFO:tasks.workunit.client.0.smithi167.stderr: Removing snap: 100% complete...done. 2023-10-05T21:09:55.672 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm rbd2/clone 2023-10-05T21:09:55.761 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:09:55.759+0000 7f6d58503700 0 -- 172.21.15.167:0/1385623743 >> [v2:172.21.15.167:6800/144105,v1:172.21.15.167:6801/144105] conn(0x55b2adad2de0 msgr2=0x55b2adad5260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:09:55.765 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:09:55.768 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap unprotect test1@s1 2023-10-05T21:09:55.811 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap rm test1@s1 2023-10-05T21:09:56.668 INFO:tasks.workunit.client.0.smithi167.stderr: Removing snap: 100% complete...done. 2023-10-05T21:09:56.684 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test1 2023-10-05T21:09:56.792 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:09:56.790+0000 7f6da4161700 0 -- 172.21.15.167:0/2565853233 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f6d88008740 msgr2=0x7f6d8800abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:09:56.792 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:09:56.795 INFO:tasks.workunit.client.0.smithi167.stderr:+ ceph osd pool rm rbd2 rbd2 --yes-i-really-really-mean-it 2023-10-05T21:09:57.793 INFO:tasks.workunit.client.0.smithi167.stderr:pool 'rbd2' does not exist 2023-10-05T21:09:57.803 INFO:tasks.workunit.client.0.smithi167.stderr:+ test_trash 2023-10-05T21:09:57.803 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'testing trash...' 2023-10-05T21:09:57.803 INFO:tasks.workunit.client.0.smithi167.stdout:testing trash... 2023-10-05T21:09:57.804 INFO:tasks.workunit.client.0.smithi167.stderr:+ remove_images 2023-10-05T21:09:57.804 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:57.886 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:57.967 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:58.047 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:58.126 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:58.205 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:58.282 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:58.360 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:58.439 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:58.517 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:58.595 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:58.678 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:58.757 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:58.837 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:58.916 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:58.995 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:59.077 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:59.157 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:09:59.235 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 2 -s 1 test1 2023-10-05T21:09:59.709 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 2 -s 1 test2 2023-10-05T21:09:59.709 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls 2023-10-05T21:09:59.709 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep test1 2023-10-05T21:09:59.709 INFO:tasks.workunit.client.0.smithi167.stdout:test1 2023-10-05T21:09:59.710 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls 2023-10-05T21:09:59.710 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep test2 2023-10-05T21:09:59.710 INFO:tasks.workunit.client.0.smithi167.stdout:test2 2023-10-05T21:09:59.711 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls 2023-10-05T21:09:59.711 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:09:59.711 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 2 2023-10-05T21:09:59.711 INFO:tasks.workunit.client.0.smithi167.stdout:2 2023-10-05T21:09:59.712 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls -l 2023-10-05T21:09:59.712 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'test1.*2.*' 2023-10-05T21:09:59.712 INFO:tasks.workunit.client.0.smithi167.stdout:test1 1 MiB 2 2023-10-05T21:09:59.712 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls -l 2023-10-05T21:09:59.712 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'test2.*2.*' 2023-10-05T21:09:59.713 INFO:tasks.workunit.client.0.smithi167.stdout:test2 1 MiB 2 2023-10-05T21:09:59.713 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash mv test1 2023-10-05T21:09:59.713 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls 2023-10-05T21:09:59.713 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep test2 2023-10-05T21:09:59.714 INFO:tasks.workunit.client.0.smithi167.stdout:test2 2023-10-05T21:09:59.714 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls 2023-10-05T21:09:59.714 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:09:59.714 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 1 2023-10-05T21:09:59.715 INFO:tasks.workunit.client.0.smithi167.stdout:1 2023-10-05T21:09:59.715 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls -l 2023-10-05T21:09:59.715 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'test2.*2.*' 2023-10-05T21:09:59.716 INFO:tasks.workunit.client.0.smithi167.stdout:test2 1 MiB 2 2023-10-05T21:09:59.716 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:09:59.716 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep test1 2023-10-05T21:09:59.716 INFO:tasks.workunit.client.0.smithi167.stdout:2268113df6ba test1 2023-10-05T21:09:59.717 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:09:59.717 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:09:59.717 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 1 2023-10-05T21:09:59.717 INFO:tasks.workunit.client.0.smithi167.stdout:1 2023-10-05T21:09:59.718 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls -l 2023-10-05T21:09:59.718 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'test1.*USER.*' 2023-10-05T21:09:59.741 INFO:tasks.workunit.client.0.smithi167.stdout:2268113df6ba test1 USER Thu Oct 5 21:09:59 2023 expired at Thu Oct 5 21:09:59 2023 2023-10-05T21:09:59.741 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls -l 2023-10-05T21:09:59.741 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep -v 'protected until' 2023-10-05T21:09:59.775 INFO:tasks.workunit.client.0.smithi167.stdout:ID NAME SOURCE DELETED_AT STATUS PARENT 2023-10-05T21:09:59.775 INFO:tasks.workunit.client.0.smithi167.stdout:2268113df6ba test1 USER Thu Oct 5 21:09:59 2023 expired at Thu Oct 5 21:09:59 2023 2023-10-05T21:09:59.776 INFO:tasks.workunit.client.0.smithi167.stderr:++ rbd trash ls 2023-10-05T21:09:59.776 INFO:tasks.workunit.client.0.smithi167.stderr:++ cut -d ' ' -f 1 2023-10-05T21:09:59.803 INFO:tasks.workunit.client.0.smithi167.stderr:+ ID=2268113df6ba 2023-10-05T21:09:59.828 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash rm 2268113df6ba 2023-10-05T21:09:59.862 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:09:59.864 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash mv test2 2023-10-05T21:09:59.924 INFO:tasks.workunit.client.0.smithi167.stderr:++ rbd trash ls 2023-10-05T21:09:59.925 INFO:tasks.workunit.client.0.smithi167.stderr:++ cut -d ' ' -f 1 2023-10-05T21:09:59.952 INFO:tasks.workunit.client.0.smithi167.stderr:+ ID=226b578f554d 2023-10-05T21:09:59.952 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info --image-id 226b578f554d 2023-10-05T21:09:59.952 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'rbd image '\''test2'\''' 2023-10-05T21:09:59.987 INFO:tasks.workunit.client.0.smithi167.stdout:rbd image 'test2': 2023-10-05T21:09:59.987 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash restore 226b578f554d 2023-10-05T21:10:00.027 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls 2023-10-05T21:10:00.027 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep test2 2023-10-05T21:10:00.053 INFO:tasks.workunit.client.0.smithi167.stdout:test2 2023-10-05T21:10:00.054 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls 2023-10-05T21:10:00.054 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:00.054 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 1 2023-10-05T21:10:00.081 INFO:tasks.workunit.client.0.smithi167.stdout:1 2023-10-05T21:10:00.081 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls -l 2023-10-05T21:10:00.081 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'test2.*2.*' 2023-10-05T21:10:00.117 INFO:tasks.workunit.client.0.smithi167.stdout:test2 1 MiB 2 2023-10-05T21:10:00.117 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash mv test2 --expires-at '3600 sec' 2023-10-05T21:10:00.174 INFO:tasks.workunit.client.0.smithi167.stdout:rbd: image test2 will expire at 2023-10-05T22:10:00.145461+0000 2023-10-05T21:10:00.177 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:00.177 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep test2 2023-10-05T21:10:00.204 INFO:tasks.workunit.client.0.smithi167.stdout:226b578f554d test2 2023-10-05T21:10:00.205 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:00.205 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:00.205 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 1 2023-10-05T21:10:00.231 INFO:tasks.workunit.client.0.smithi167.stdout:1 2023-10-05T21:10:00.232 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls -l 2023-10-05T21:10:00.232 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'test2.*USER.*protected until' 2023-10-05T21:10:00.266 INFO:tasks.workunit.client.0.smithi167.stdout:226b578f554d test2 USER Thu Oct 5 21:10:00 2023 protected until Thu Oct 5 22:10:00 2023 2023-10-05T21:10:00.267 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash rm 226b578f554d 2023-10-05T21:10:00.267 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'Deferment time has not expired' 2023-10-05T21:10:00.295 INFO:tasks.workunit.client.0.smithi167.stdout:Deferment time has not expired, please use --force if you 2023-10-05T21:10:00.291+0000 7f5bdae98540 -1 librbd::api::Trash: remove: error: deferment time has not expired. 2023-10-05T21:10:00.295 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash rm --image-id 226b578f554d --force 2023-10-05T21:10:00.357 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:10:00.360 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 2 -s 1 test1 2023-10-05T21:10:00.403 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap create test1@snap1 2023-10-05T21:10:00.695 INFO:tasks.workunit.client.0.smithi167.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-05T21:10:00.703 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap protect test1@snap1 2023-10-05T21:10:00.744 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash mv test1 2023-10-05T21:10:00.801 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:00.801 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep test1 2023-10-05T21:10:00.827 INFO:tasks.workunit.client.0.smithi167.stdout:22c167351c46 test1 2023-10-05T21:10:00.828 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:00.828 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:00.828 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 1 2023-10-05T21:10:00.855 INFO:tasks.workunit.client.0.smithi167.stdout:1 2023-10-05T21:10:00.855 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls -l 2023-10-05T21:10:00.855 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'test1.*USER.*' 2023-10-05T21:10:00.890 INFO:tasks.workunit.client.0.smithi167.stdout:22c167351c46 test1 USER Thu Oct 5 21:10:00 2023 expired at Thu Oct 5 21:10:00 2023 2023-10-05T21:10:00.890 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls -l 2023-10-05T21:10:00.891 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep -v 'protected until' 2023-10-05T21:10:00.926 INFO:tasks.workunit.client.0.smithi167.stdout:ID NAME SOURCE DELETED_AT STATUS PARENT 2023-10-05T21:10:00.926 INFO:tasks.workunit.client.0.smithi167.stdout:22c167351c46 test1 USER Thu Oct 5 21:10:00 2023 expired at Thu Oct 5 21:10:00 2023 2023-10-05T21:10:00.926 INFO:tasks.workunit.client.0.smithi167.stderr:++ rbd trash ls 2023-10-05T21:10:00.926 INFO:tasks.workunit.client.0.smithi167.stderr:++ cut -d ' ' -f 1 2023-10-05T21:10:00.954 INFO:tasks.workunit.client.0.smithi167.stderr:+ ID=22c167351c46 2023-10-05T21:10:00.955 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap ls --image-id 22c167351c46 2023-10-05T21:10:00.955 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep -v SNAPID 2023-10-05T21:10:00.955 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:00.955 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 1 2023-10-05T21:10:00.990 INFO:tasks.workunit.client.0.smithi167.stdout:1 2023-10-05T21:10:00.991 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap ls --image-id 22c167351c46 2023-10-05T21:10:00.991 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '.*snap1.*' 2023-10-05T21:10:01.026 INFO:tasks.workunit.client.0.smithi167.stdout: 34 snap1 1 MiB yes Thu Oct 5 21:10:00 2023 2023-10-05T21:10:01.026 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap unprotect --image-id 22c167351c46 --snap snap1 2023-10-05T21:10:01.071 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap rm --image-id 22c167351c46 --snap snap1 2023-10-05T21:10:01.612 INFO:tasks.workunit.client.0.smithi167.stderr: Removing snap: 100% complete...done. 2023-10-05T21:10:01.629 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap ls --image-id 22c167351c46 2023-10-05T21:10:01.629 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep -v SNAPID 2023-10-05T21:10:01.629 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:01.629 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 0 2023-10-05T21:10:01.665 INFO:tasks.workunit.client.0.smithi167.stdout:0 2023-10-05T21:10:01.665 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash restore 22c167351c46 2023-10-05T21:10:01.706 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap create test1@snap1 2023-10-05T21:10:02.699 INFO:tasks.workunit.client.0.smithi167.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-05T21:10:02.709 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap create test1@snap2 2023-10-05T21:10:03.698 INFO:tasks.workunit.client.0.smithi167.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-05T21:10:03.714 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap ls --image-id 22c167351c46 2023-10-05T21:10:03.714 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep -v SNAPID 2023-10-05T21:10:03.714 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:03.715 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 2 2023-10-05T21:10:03.755 INFO:tasks.workunit.client.0.smithi167.stdout:2 2023-10-05T21:10:03.755 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap purge --image-id 22c167351c46 2023-10-05T21:10:05.696 INFO:tasks.workunit.client.0.smithi167.stderr: Removing all snapshots: 50% complete... Removing all snapshots: 100% complete... Removing all snapshots: 100% complete...done. 2023-10-05T21:10:05.711 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap ls --image-id 22c167351c46 2023-10-05T21:10:05.711 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep -v SNAPID 2023-10-05T21:10:05.711 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:05.711 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 0 2023-10-05T21:10:05.751 INFO:tasks.workunit.client.0.smithi167.stdout:0 2023-10-05T21:10:05.752 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm --rbd_move_to_trash_on_remove=true --rbd_move_to_trash_on_remove_expire_seconds=3600 test1 2023-10-05T21:10:05.810 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:10:05.813 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:05.813 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep test1 2023-10-05T21:10:05.840 INFO:tasks.workunit.client.0.smithi167.stdout:22c167351c46 test1 2023-10-05T21:10:05.840 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:05.841 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:05.841 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 1 2023-10-05T21:10:05.868 INFO:tasks.workunit.client.0.smithi167.stdout:1 2023-10-05T21:10:05.868 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls -l 2023-10-05T21:10:05.868 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'test1.*USER.*protected until' 2023-10-05T21:10:05.903 INFO:tasks.workunit.client.0.smithi167.stdout:22c167351c46 test1 USER Thu Oct 5 21:10:05 2023 protected until Thu Oct 5 22:10:05 2023 2023-10-05T21:10:05.904 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash rm 22c167351c46 2023-10-05T21:10:05.904 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'Deferment time has not expired' 2023-10-05T21:10:05.932 INFO:tasks.workunit.client.0.smithi167.stdout:Deferment time has not expired, please use --force if you really want to remove the image 2023-10-05T21:10:05.933 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash rm --image-id 22c167351c46 --force 2023-10-05T21:10:05.993 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:10:05.995 INFO:tasks.workunit.client.0.smithi167.stderr:+ remove_images 2023-10-05T21:10:05.995 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:06.082 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:06.162 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:06.245 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:06.328 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:06.409 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:06.487 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:06.566 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:06.650 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:06.728 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:06.809 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:06.891 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:06.973 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:07.054 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:07.133 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:07.211 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:07.290 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:07.369 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:07.448 INFO:tasks.workunit.client.0.smithi167.stderr:+ test_purge 2023-10-05T21:10:07.449 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'testing trash purge...' 2023-10-05T21:10:07.449 INFO:tasks.workunit.client.0.smithi167.stdout:testing trash purge... 2023-10-05T21:10:07.449 INFO:tasks.workunit.client.0.smithi167.stderr:+ remove_images 2023-10-05T21:10:07.449 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:07.528 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:07.608 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:07.687 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:07.766 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:07.846 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:07.926 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:08.005 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:08.089 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:08.166 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:08.244 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:08.323 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:08.401 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:08.482 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:08.562 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:08.641 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:08.722 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:08.802 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:08.882 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:08.882 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:08.882 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 0 2023-10-05T21:10:08.909 INFO:tasks.workunit.client.0.smithi167.stdout:0 2023-10-05T21:10:08.909 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash purge 2023-10-05T21:10:08.934 INFO:tasks.workunit.client.0.smithi167.stderr: Removing images: 100% complete...done. 2023-10-05T21:10:08.936 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 2 --size 256 testimg1 2023-10-05T21:10:08.984 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 2 --size 256 testimg2 2023-10-05T21:10:09.043 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash mv testimg1 2023-10-05T21:10:09.101 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash mv testimg2 2023-10-05T21:10:09.161 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:09.161 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:09.161 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 2 2023-10-05T21:10:09.187 INFO:tasks.workunit.client.0.smithi167.stdout:2 2023-10-05T21:10:09.187 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash purge 2023-10-05T21:10:09.273 INFO:tasks.workunit.client.0.smithi167.stderr: Removing images: 50% complete... Removing images: 100% complete... Removing images: 100% complete...done. 2023-10-05T21:10:09.276 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:09.276 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:09.276 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 0 2023-10-05T21:10:09.303 INFO:tasks.workunit.client.0.smithi167.stdout:0 2023-10-05T21:10:09.304 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 2 --size 256 testimg1 2023-10-05T21:10:09.348 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 2 --size 256 testimg2 2023-10-05T21:10:09.397 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash mv testimg1 --expires-at '1 hour' 2023-10-05T21:10:09.462 INFO:tasks.workunit.client.0.smithi167.stdout:rbd: image testimg1 will expire at 2023-10-05T22:10:09.426012+0000 2023-10-05T21:10:09.465 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash mv testimg2 --expires-at '3 hours' 2023-10-05T21:10:09.518 INFO:tasks.workunit.client.0.smithi167.stdout:rbd: image testimg2 will expire at 2023-10-06T00:10:09.493574+0000 2023-10-05T21:10:09.522 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:09.522 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:09.522 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 2 2023-10-05T21:10:09.549 INFO:tasks.workunit.client.0.smithi167.stdout:2 2023-10-05T21:10:09.550 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash purge 2023-10-05T21:10:09.575 INFO:tasks.workunit.client.0.smithi167.stderr: Removing images: 100% complete...done. 2023-10-05T21:10:09.577 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:09.578 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:09.578 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 2 2023-10-05T21:10:09.605 INFO:tasks.workunit.client.0.smithi167.stdout:2 2023-10-05T21:10:09.605 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash purge --expired-before 'now + 2 hours' 2023-10-05T21:10:09.684 INFO:tasks.workunit.client.0.smithi167.stderr: Removing images: 100% complete... Removing images: 100% complete...done. 2023-10-05T21:10:09.687 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:09.687 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:09.687 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 1 2023-10-05T21:10:09.714 INFO:tasks.workunit.client.0.smithi167.stdout:1 2023-10-05T21:10:09.715 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:09.715 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep testimg2 2023-10-05T21:10:09.742 INFO:tasks.workunit.client.0.smithi167.stdout:240561087c0 testimg2 2023-10-05T21:10:09.742 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash purge --expired-before 'now + 4 hours' 2023-10-05T21:10:09.803 INFO:tasks.workunit.client.0.smithi167.stderr: Removing images: 100% complete... Removing images: 100% complete...done. 2023-10-05T21:10:09.806 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:09.806 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:09.806 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 0 2023-10-05T21:10:09.833 INFO:tasks.workunit.client.0.smithi167.stdout:0 2023-10-05T21:10:09.833 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 2 --size 256 testimg1 2023-10-05T21:10:09.877 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap create testimg1@snap 2023-10-05T21:10:10.716 INFO:tasks.workunit.client.0.smithi167.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-05T21:10:10.726 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 2 --size 256 testimg2 2023-10-05T21:10:10.774 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 2 --size 256 testimg3 2023-10-05T21:10:10.822 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash mv testimg1 2023-10-05T21:10:10.879 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash mv testimg2 2023-10-05T21:10:10.958 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash mv testimg3 2023-10-05T21:10:11.016 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:11.016 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:11.017 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 3 2023-10-05T21:10:11.043 INFO:tasks.workunit.client.0.smithi167.stdout:3 2023-10-05T21:10:11.044 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash purge 2023-10-05T21:10:11.044 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'some expired images could not be removed' 2023-10-05T21:10:11.178 INFO:tasks.workunit.client.0.smithi167.stdout:rbd: some expired images could not be removed 2023-10-05T21:10:11.179 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:11.179 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:11.179 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 1 2023-10-05T21:10:11.206 INFO:tasks.workunit.client.0.smithi167.stdout:1 2023-10-05T21:10:11.207 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:11.207 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep testimg1 2023-10-05T21:10:11.233 INFO:tasks.workunit.client.0.smithi167.stdout:24253772e8e6 testimg1 2023-10-05T21:10:11.234 INFO:tasks.workunit.client.0.smithi167.stderr:++ rbd trash ls 2023-10-05T21:10:11.234 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{ print $1 }' 2023-10-05T21:10:11.262 INFO:tasks.workunit.client.0.smithi167.stderr:+ ID=24253772e8e6 2023-10-05T21:10:11.262 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap purge --image-id 24253772e8e6 2023-10-05T21:10:11.614 INFO:tasks.workunit.client.0.smithi167.stderr: Removing all snapshots: 100% complete... Removing all snapshots: 100% complete...done. 2023-10-05T21:10:11.635 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash purge 2023-10-05T21:10:11.700 INFO:tasks.workunit.client.0.smithi167.stderr: Removing images: 100% complete... Removing images: 100% complete...done. 2023-10-05T21:10:11.703 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:11.703 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:11.703 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 0 2023-10-05T21:10:11.729 INFO:tasks.workunit.client.0.smithi167.stdout:0 2023-10-05T21:10:11.730 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 2 --size 256 testimg1 2023-10-05T21:10:11.775 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 2 --size 256 testimg2 2023-10-05T21:10:11.824 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap create testimg2@snap 2023-10-05T21:10:12.718 INFO:tasks.workunit.client.0.smithi167.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-05T21:10:12.727 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 2 --size 256 testimg3 2023-10-05T21:10:12.776 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash mv testimg1 2023-10-05T21:10:12.844 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash mv testimg2 2023-10-05T21:10:12.901 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash mv testimg3 2023-10-05T21:10:12.957 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:12.957 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:12.957 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 3 2023-10-05T21:10:12.983 INFO:tasks.workunit.client.0.smithi167.stdout:3 2023-10-05T21:10:12.984 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash purge 2023-10-05T21:10:12.984 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'some expired images could not be removed' 2023-10-05T21:10:13.120 INFO:tasks.workunit.client.0.smithi167.stdout:rbd: some expired images could not be removed 2023-10-05T21:10:13.120 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:13.121 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:13.121 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 1 2023-10-05T21:10:13.147 INFO:tasks.workunit.client.0.smithi167.stdout:1 2023-10-05T21:10:13.148 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:13.148 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep testimg2 2023-10-05T21:10:13.174 INFO:tasks.workunit.client.0.smithi167.stdout:2455feec0ef9 testimg2 2023-10-05T21:10:13.175 INFO:tasks.workunit.client.0.smithi167.stderr:++ rbd trash ls 2023-10-05T21:10:13.175 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{ print $1 }' 2023-10-05T21:10:13.203 INFO:tasks.workunit.client.0.smithi167.stderr:+ ID=2455feec0ef9 2023-10-05T21:10:13.203 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap purge --image-id 2455feec0ef9 2023-10-05T21:10:13.717 INFO:tasks.workunit.client.0.smithi167.stderr: Removing all snapshots: 100% complete... Removing all snapshots: 100% complete...done. 2023-10-05T21:10:13.729 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash purge 2023-10-05T21:10:13.791 INFO:tasks.workunit.client.0.smithi167.stderr: Removing images: 100% complete... Removing images: 100% complete...done. 2023-10-05T21:10:13.794 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:13.795 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:13.795 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 0 2023-10-05T21:10:13.821 INFO:tasks.workunit.client.0.smithi167.stdout:0 2023-10-05T21:10:13.822 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 2 --size 256 testimg1 2023-10-05T21:10:13.868 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 2 --size 256 testimg2 2023-10-05T21:10:13.914 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 2 --size 256 testimg3 2023-10-05T21:10:13.963 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap create testimg3@snap 2023-10-05T21:10:14.731 INFO:tasks.workunit.client.0.smithi167.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-05T21:10:14.741 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash mv testimg1 2023-10-05T21:10:14.813 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash mv testimg2 2023-10-05T21:10:14.879 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash mv testimg3 2023-10-05T21:10:14.945 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:14.945 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:14.945 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 3 2023-10-05T21:10:14.972 INFO:tasks.workunit.client.0.smithi167.stdout:3 2023-10-05T21:10:14.972 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash purge 2023-10-05T21:10:14.972 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'some expired images could not be removed' 2023-10-05T21:10:15.122 INFO:tasks.workunit.client.0.smithi167.stdout:rbd: some expired images could not be removed 2023-10-05T21:10:15.123 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:15.123 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:15.123 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 1 2023-10-05T21:10:15.150 INFO:tasks.workunit.client.0.smithi167.stdout:1 2023-10-05T21:10:15.151 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:15.151 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep testimg3 2023-10-05T21:10:15.178 INFO:tasks.workunit.client.0.smithi167.stdout:24854414049 testimg3 2023-10-05T21:10:15.179 INFO:tasks.workunit.client.0.smithi167.stderr:++ rbd trash ls 2023-10-05T21:10:15.179 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{ print $1 }' 2023-10-05T21:10:15.207 INFO:tasks.workunit.client.0.smithi167.stderr:+ ID=24854414049 2023-10-05T21:10:15.207 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap purge --image-id 24854414049 2023-10-05T21:10:15.730 INFO:tasks.workunit.client.0.smithi167.stderr: Removing all snapshots: 100% complete... Removing all snapshots: 100% complete...done. 2023-10-05T21:10:15.741 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash purge 2023-10-05T21:10:15.802 INFO:tasks.workunit.client.0.smithi167.stderr: Removing images: 100% complete... Removing images: 100% complete...done. 2023-10-05T21:10:15.805 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:15.805 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:15.805 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 0 2023-10-05T21:10:15.832 INFO:tasks.workunit.client.0.smithi167.stdout:0 2023-10-05T21:10:15.832 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 2 --size 256 testimg1 2023-10-05T21:10:15.877 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap create testimg1@snap 2023-10-05T21:10:16.627 INFO:tasks.workunit.client.0.smithi167.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-05T21:10:16.637 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd clone --rbd-default-clone-format=2 testimg1@snap testimg2 2023-10-05T21:10:16.720 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap rm testimg1@snap 2023-10-05T21:10:16.764 INFO:tasks.workunit.client.0.smithi167.stderr: Removing snap: 100% complete...done. 2023-10-05T21:10:16.772 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 2 --size 256 testimg3 2023-10-05T21:10:16.818 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap create testimg2@snap 2023-10-05T21:10:17.748 INFO:tasks.workunit.client.0.smithi167.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-05T21:10:17.759 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd clone --rbd-default-clone-format=2 testimg2@snap testimg4 2023-10-05T21:10:17.847 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd clone --rbd-default-clone-format=2 testimg2@snap testimg5 2023-10-05T21:10:17.929 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap rm testimg2@snap 2023-10-05T21:10:17.981 INFO:tasks.workunit.client.0.smithi167.stderr: Removing snap: 100% complete...done. 2023-10-05T21:10:17.990 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap create testimg4@snap 2023-10-05T21:10:18.749 INFO:tasks.workunit.client.0.smithi167.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-05T21:10:18.758 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd clone --rbd-default-clone-format=2 testimg4@snap testimg6 2023-10-05T21:10:18.844 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap rm testimg4@snap 2023-10-05T21:10:18.903 INFO:tasks.workunit.client.0.smithi167.stderr: Removing snap: 100% complete...done. 2023-10-05T21:10:18.911 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash mv testimg1 2023-10-05T21:10:18.970 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash mv testimg2 2023-10-05T21:10:19.029 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash mv testimg3 2023-10-05T21:10:19.193 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash mv testimg4 2023-10-05T21:10:19.193 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:19.193 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:19.193 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 4 2023-10-05T21:10:19.194 INFO:tasks.workunit.client.0.smithi167.stdout:4 2023-10-05T21:10:19.194 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash purge 2023-10-05T21:10:19.194 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'some expired images could not be removed' 2023-10-05T21:10:19.388 INFO:tasks.workunit.client.0.smithi167.stdout:rbd: some expired images could not be removed 2023-10-05T21:10:19.389 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:19.389 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:19.389 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 3 2023-10-05T21:10:19.415 INFO:tasks.workunit.client.0.smithi167.stdout:3 2023-10-05T21:10:19.416 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:19.416 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep testimg1 2023-10-05T21:10:19.442 INFO:tasks.workunit.client.0.smithi167.stdout:24abdbb8522c testimg1 2023-10-05T21:10:19.443 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:19.443 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep testimg2 2023-10-05T21:10:19.470 INFO:tasks.workunit.client.0.smithi167.stdout:24b16b1f3a5e testimg2 2023-10-05T21:10:19.470 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:19.470 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep testimg4 2023-10-05T21:10:19.497 INFO:tasks.workunit.client.0.smithi167.stdout:24bdc8a6b0b8 testimg4 2023-10-05T21:10:19.497 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash mv testimg6 2023-10-05T21:10:19.560 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:19.560 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:19.560 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 4 2023-10-05T21:10:19.586 INFO:tasks.workunit.client.0.smithi167.stdout:4 2023-10-05T21:10:19.587 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash purge 2023-10-05T21:10:19.587 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'some expired images could not be removed' 2023-10-05T21:10:19.923 INFO:tasks.workunit.client.0.smithi167.stdout:rbd: some expired images could not be removed 2023-10-05T21:10:19.924 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:19.924 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:19.924 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 2 2023-10-05T21:10:19.950 INFO:tasks.workunit.client.0.smithi167.stdout:2 2023-10-05T21:10:19.951 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:19.951 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep testimg1 2023-10-05T21:10:19.977 INFO:tasks.workunit.client.0.smithi167.stdout:24abdbb8522c testimg1 2023-10-05T21:10:19.978 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:19.978 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep testimg2 2023-10-05T21:10:20.005 INFO:tasks.workunit.client.0.smithi167.stdout:24b16b1f3a5e testimg2 2023-10-05T21:10:20.005 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash mv testimg5 2023-10-05T21:10:20.069 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:20.069 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:20.069 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 3 2023-10-05T21:10:20.096 INFO:tasks.workunit.client.0.smithi167.stdout:3 2023-10-05T21:10:20.096 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash purge 2023-10-05T21:10:20.187 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:10:20.186+0000 7f3ae60dc700 0 -- 172.21.15.167:0/3302803401 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x7f3ac0054af0 msgr2=0x7f3ac0056ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:10:20.196 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:10:20.195+0000 7f3ae60dc700 0 -- 172.21.15.167:0/3302803401 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x7f3ac803a6c0 msgr2=0x7f3ac803cae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:10:21.682 INFO:tasks.workunit.client.0.smithi167.stderr: Removing images: 33% complete... Removing images: 66% complete... Removing images: 100% complete... Removing images: 100% complete...done. 2023-10-05T21:10:21.685 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:21.685 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:21.686 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 0 2023-10-05T21:10:21.713 INFO:tasks.workunit.client.0.smithi167.stdout:0 2023-10-05T21:10:21.713 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 2 --size 256 testimg1 2023-10-05T21:10:21.759 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap create testimg1@snap 2023-10-05T21:10:22.771 INFO:tasks.workunit.client.0.smithi167.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-05T21:10:22.781 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd clone --rbd-default-clone-format=2 testimg1@snap testimg2 2023-10-05T21:10:22.860 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap rm testimg1@snap 2023-10-05T21:10:22.903 INFO:tasks.workunit.client.0.smithi167.stderr: Removing snap: 100% complete...done. 2023-10-05T21:10:22.911 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 2 --size 256 testimg3 2023-10-05T21:10:22.959 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap create testimg3@snap 2023-10-05T21:10:23.776 INFO:tasks.workunit.client.0.smithi167.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-05T21:10:23.794 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap create testimg2@snap 2023-10-05T21:10:24.788 INFO:tasks.workunit.client.0.smithi167.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-05T21:10:24.798 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd clone --rbd-default-clone-format=2 testimg2@snap testimg4 2023-10-05T21:10:24.884 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd clone --rbd-default-clone-format=2 testimg2@snap testimg5 2023-10-05T21:10:24.964 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap rm testimg2@snap 2023-10-05T21:10:25.012 INFO:tasks.workunit.client.0.smithi167.stderr: Removing snap: 100% complete...done. 2023-10-05T21:10:25.020 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap create testimg4@snap 2023-10-05T21:10:25.797 INFO:tasks.workunit.client.0.smithi167.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-05T21:10:25.810 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd clone --rbd-default-clone-format=2 testimg4@snap testimg6 2023-10-05T21:10:25.901 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap rm testimg4@snap 2023-10-05T21:10:25.958 INFO:tasks.workunit.client.0.smithi167.stderr: Removing snap: 100% complete...done. 2023-10-05T21:10:25.967 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash mv testimg1 2023-10-05T21:10:26.024 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash mv testimg2 2023-10-05T21:10:26.082 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash mv testimg3 2023-10-05T21:10:26.142 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash mv testimg4 2023-10-05T21:10:26.205 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:26.205 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:26.206 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 4 2023-10-05T21:10:26.232 INFO:tasks.workunit.client.0.smithi167.stdout:4 2023-10-05T21:10:26.233 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash purge 2023-10-05T21:10:26.233 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'some expired images could not be removed' 2023-10-05T21:10:26.362 INFO:tasks.workunit.client.0.smithi167.stdout:rbd: some expired images could not be removed 2023-10-05T21:10:26.363 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:26.363 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:26.363 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 4 2023-10-05T21:10:26.389 INFO:tasks.workunit.client.0.smithi167.stdout:4 2023-10-05T21:10:26.390 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash mv testimg6 2023-10-05T21:10:26.447 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:26.447 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:26.447 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 5 2023-10-05T21:10:26.473 INFO:tasks.workunit.client.0.smithi167.stdout:5 2023-10-05T21:10:26.474 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash purge 2023-10-05T21:10:26.474 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'some expired images could not be removed' 2023-10-05T21:10:27.839 INFO:tasks.workunit.client.0.smithi167.stdout:rbd: some expired images could not be removed 2023-10-05T21:10:27.839 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:27.839 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:27.840 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 3 2023-10-05T21:10:27.866 INFO:tasks.workunit.client.0.smithi167.stdout:3 2023-10-05T21:10:27.867 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:27.867 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep testimg1 2023-10-05T21:10:27.893 INFO:tasks.workunit.client.0.smithi167.stdout:250b219e3c6d testimg1 2023-10-05T21:10:27.894 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:27.894 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep testimg2 2023-10-05T21:10:27.921 INFO:tasks.workunit.client.0.smithi167.stdout:2511b13eb812 testimg2 2023-10-05T21:10:27.922 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:27.922 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep testimg3 2023-10-05T21:10:27.948 INFO:tasks.workunit.client.0.smithi167.stdout:2517d74e8b97 testimg3 2023-10-05T21:10:27.949 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash mv testimg5 2023-10-05T21:10:28.012 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:28.012 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:28.012 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 4 2023-10-05T21:10:28.039 INFO:tasks.workunit.client.0.smithi167.stdout:4 2023-10-05T21:10:28.039 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash purge 2023-10-05T21:10:28.039 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'some expired images could not be removed' 2023-10-05T21:10:29.940 INFO:tasks.workunit.client.0.smithi167.stdout:rbd: some expired images could not be removed 2023-10-05T21:10:29.940 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:29.941 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:29.941 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 1 2023-10-05T21:10:29.967 INFO:tasks.workunit.client.0.smithi167.stdout:1 2023-10-05T21:10:29.968 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:29.968 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep testimg3 2023-10-05T21:10:29.994 INFO:tasks.workunit.client.0.smithi167.stdout:2517d74e8b97 testimg3 2023-10-05T21:10:29.995 INFO:tasks.workunit.client.0.smithi167.stderr:++ rbd trash ls 2023-10-05T21:10:29.995 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{ print $1 }' 2023-10-05T21:10:30.025 INFO:tasks.workunit.client.0.smithi167.stderr:+ ID=2517d74e8b97 2023-10-05T21:10:30.026 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap purge --image-id 2517d74e8b97 2023-10-05T21:10:30.808 INFO:tasks.workunit.client.0.smithi167.stderr: Removing all snapshots: 100% complete... Removing all snapshots: 100% complete...done. 2023-10-05T21:10:30.829 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash purge 2023-10-05T21:10:30.893 INFO:tasks.workunit.client.0.smithi167.stderr: Removing images: 100% complete... Removing images: 100% complete...done. 2023-10-05T21:10:30.896 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:30.896 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:30.896 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 0 2023-10-05T21:10:30.924 INFO:tasks.workunit.client.0.smithi167.stdout:0 2023-10-05T21:10:30.924 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 2 --size 256 testimg1 2023-10-05T21:10:30.970 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap create testimg1@snap 2023-10-05T21:10:31.627 INFO:tasks.workunit.client.0.smithi167.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-05T21:10:31.639 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd clone --rbd-default-clone-format=2 testimg1@snap testimg2 2023-10-05T21:10:31.710 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap rm testimg1@snap 2023-10-05T21:10:31.751 INFO:tasks.workunit.client.0.smithi167.stderr: Removing snap: 100% complete...done. 2023-10-05T21:10:31.760 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 2 --size 256 testimg3 2023-10-05T21:10:31.805 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap create testimg2@snap 2023-10-05T21:10:32.825 INFO:tasks.workunit.client.0.smithi167.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-05T21:10:32.836 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd clone --rbd-default-clone-format=2 testimg2@snap testimg4 2023-10-05T21:10:32.924 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd clone --rbd-default-clone-format=2 testimg2@snap testimg5 2023-10-05T21:10:33.001 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap rm testimg2@snap 2023-10-05T21:10:33.051 INFO:tasks.workunit.client.0.smithi167.stderr: Removing snap: 100% complete...done. 2023-10-05T21:10:33.059 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap create testimg4@snap 2023-10-05T21:10:33.824 INFO:tasks.workunit.client.0.smithi167.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-05T21:10:33.832 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd clone --rbd-default-clone-format=2 testimg4@snap testimg6 2023-10-05T21:10:33.910 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap rm testimg4@snap 2023-10-05T21:10:33.964 INFO:tasks.workunit.client.0.smithi167.stderr: Removing snap: 100% complete...done. 2023-10-05T21:10:33.973 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm --rbd_move_parent_to_trash_on_remove=true testimg1 2023-10-05T21:10:34.051 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:10:34.054 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm --rbd_move_parent_to_trash_on_remove=true testimg2 2023-10-05T21:10:34.136 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:10:34.139 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash mv testimg3 2023-10-05T21:10:34.201 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm --rbd_move_parent_to_trash_on_remove=true testimg4 2023-10-05T21:10:34.278 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:10:34.281 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:34.281 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:34.281 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 4 2023-10-05T21:10:34.308 INFO:tasks.workunit.client.0.smithi167.stdout:4 2023-10-05T21:10:34.308 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash purge 2023-10-05T21:10:34.308 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'some expired images could not be removed' 2023-10-05T21:10:34.518 INFO:tasks.workunit.client.0.smithi167.stdout:rbd: some expired images could not be removed 2023-10-05T21:10:34.519 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:34.519 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:34.519 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 3 2023-10-05T21:10:34.547 INFO:tasks.workunit.client.0.smithi167.stdout:3 2023-10-05T21:10:34.547 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:34.547 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep testimg1 2023-10-05T21:10:34.573 INFO:tasks.workunit.client.0.smithi167.stdout:2577ff585d2d testimg1 2023-10-05T21:10:34.574 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:34.574 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep testimg2 2023-10-05T21:10:34.600 INFO:tasks.workunit.client.0.smithi167.stdout:257d48c44d3 testimg2 2023-10-05T21:10:34.601 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:34.601 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep testimg4 2023-10-05T21:10:34.628 INFO:tasks.workunit.client.0.smithi167.stdout:25899e8f6e6e testimg4 2023-10-05T21:10:34.628 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash mv testimg6 2023-10-05T21:10:34.689 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:34.690 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:34.690 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 4 2023-10-05T21:10:34.716 INFO:tasks.workunit.client.0.smithi167.stdout:4 2023-10-05T21:10:34.717 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash purge 2023-10-05T21:10:34.717 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'some expired images could not be removed' 2023-10-05T21:10:35.938 INFO:tasks.workunit.client.0.smithi167.stdout:rbd: some expired images could not be removed 2023-10-05T21:10:35.939 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:35.939 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:35.939 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 2 2023-10-05T21:10:35.966 INFO:tasks.workunit.client.0.smithi167.stdout:2 2023-10-05T21:10:35.967 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:35.967 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep testimg1 2023-10-05T21:10:35.994 INFO:tasks.workunit.client.0.smithi167.stdout:2577ff585d2d testimg1 2023-10-05T21:10:35.995 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:35.995 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep testimg2 2023-10-05T21:10:36.022 INFO:tasks.workunit.client.0.smithi167.stdout:257d48c44d3 testimg2 2023-10-05T21:10:36.022 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash mv testimg5 2023-10-05T21:10:36.080 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:36.080 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:36.080 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 3 2023-10-05T21:10:36.106 INFO:tasks.workunit.client.0.smithi167.stdout:3 2023-10-05T21:10:36.107 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash purge 2023-10-05T21:10:36.200 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:10:36.199+0000 7fc239021700 0 -- 172.21.15.167:0/305471995 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x7fc218054af0 msgr2=0x7fc218056ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:10:36.210 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:10:36.208+0000 7fc239021700 0 -- 172.21.15.167:0/305471995 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x7fc21c03a720 msgr2=0x7fc21c03cb40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-05T21:10:37.694 INFO:tasks.workunit.client.0.smithi167.stderr: Removing images: 33% complete... Removing images: 66% complete... Removing images: 100% complete... Removing images: 100% complete...done. 2023-10-05T21:10:37.697 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:37.697 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:37.697 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 0 2023-10-05T21:10:37.725 INFO:tasks.workunit.client.0.smithi167.stdout:0 2023-10-05T21:10:37.725 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 2 --size 256 testimg1 2023-10-05T21:10:37.778 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap create testimg1@snap 2023-10-05T21:10:38.847 INFO:tasks.workunit.client.0.smithi167.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-05T21:10:38.857 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd clone --rbd-default-clone-format=2 testimg1@snap testimg2 2023-10-05T21:10:38.931 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap rm testimg1@snap 2023-10-05T21:10:38.974 INFO:tasks.workunit.client.0.smithi167.stderr: Removing snap: 100% complete...done. 2023-10-05T21:10:38.983 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 2 --size 256 testimg3 2023-10-05T21:10:39.027 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap create testimg3@snap 2023-10-05T21:10:39.846 INFO:tasks.workunit.client.0.smithi167.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-05T21:10:39.857 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap create testimg2@snap 2023-10-05T21:10:40.849 INFO:tasks.workunit.client.0.smithi167.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-05T21:10:41.066 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd clone --rbd-default-clone-format=2 testimg2@snap testimg4 2023-10-05T21:10:41.066 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd clone --rbd-default-clone-format=2 testimg2@snap testimg5 2023-10-05T21:10:41.066 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap rm testimg2@snap 2023-10-05T21:10:41.070 INFO:tasks.workunit.client.0.smithi167.stderr: Removing snap: 100% complete...done. 2023-10-05T21:10:41.078 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap create testimg4@snap 2023-10-05T21:10:41.628 INFO:tasks.workunit.client.0.smithi167.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-05T21:10:41.636 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd clone --rbd-default-clone-format=2 testimg4@snap testimg6 2023-10-05T21:10:41.726 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap rm testimg4@snap 2023-10-05T21:10:41.779 INFO:tasks.workunit.client.0.smithi167.stderr: Removing snap: 100% complete...done. 2023-10-05T21:10:41.786 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm --rbd_move_parent_to_trash_on_remove=true testimg1 2023-10-05T21:10:41.863 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:10:41.866 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm --rbd_move_parent_to_trash_on_remove=true testimg2 2023-10-05T21:10:41.938 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:10:41.941 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash mv testimg3 2023-10-05T21:10:42.001 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm --rbd_move_parent_to_trash_on_remove=true testimg4 2023-10-05T21:10:42.074 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:10:42.077 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:42.077 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:42.078 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 4 2023-10-05T21:10:42.105 INFO:tasks.workunit.client.0.smithi167.stdout:4 2023-10-05T21:10:42.105 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash purge 2023-10-05T21:10:42.105 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'some expired images could not be removed' 2023-10-05T21:10:42.232 INFO:tasks.workunit.client.0.smithi167.stdout:rbd: some expired images could not be removed 2023-10-05T21:10:42.233 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:42.233 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:42.234 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 4 2023-10-05T21:10:42.260 INFO:tasks.workunit.client.0.smithi167.stdout:4 2023-10-05T21:10:42.260 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash mv testimg6 2023-10-05T21:10:42.318 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:42.318 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:42.319 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 5 2023-10-05T21:10:42.345 INFO:tasks.workunit.client.0.smithi167.stdout:5 2023-10-05T21:10:42.345 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash purge 2023-10-05T21:10:42.346 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'some expired images could not be removed' 2023-10-05T21:10:42.957 INFO:tasks.workunit.client.0.smithi167.stdout:rbd: some expired images could not be removed 2023-10-05T21:10:42.958 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:42.958 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:42.958 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 3 2023-10-05T21:10:42.986 INFO:tasks.workunit.client.0.smithi167.stdout:3 2023-10-05T21:10:42.986 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:42.986 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep testimg1 2023-10-05T21:10:43.014 INFO:tasks.workunit.client.0.smithi167.stdout:25d7d51803dc testimg1 2023-10-05T21:10:43.014 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:43.014 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep testimg2 2023-10-05T21:10:43.041 INFO:tasks.workunit.client.0.smithi167.stdout:25dd7f3a13d testimg2 2023-10-05T21:10:43.042 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:43.042 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep testimg3 2023-10-05T21:10:43.069 INFO:tasks.workunit.client.0.smithi167.stdout:25e37b05efa8 testimg3 2023-10-05T21:10:43.069 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash mv testimg5 2023-10-05T21:10:43.131 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:43.131 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:43.131 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 4 2023-10-05T21:10:43.158 INFO:tasks.workunit.client.0.smithi167.stdout:4 2023-10-05T21:10:43.158 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash purge 2023-10-05T21:10:43.158 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'some expired images could not be removed' 2023-10-05T21:10:44.962 INFO:tasks.workunit.client.0.smithi167.stdout:rbd: some expired images could not be removed 2023-10-05T21:10:44.963 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:44.963 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:44.963 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 1 2023-10-05T21:10:44.990 INFO:tasks.workunit.client.0.smithi167.stdout:1 2023-10-05T21:10:44.991 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:44.991 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep testimg3 2023-10-05T21:10:45.017 INFO:tasks.workunit.client.0.smithi167.stdout:25e37b05efa8 testimg3 2023-10-05T21:10:45.018 INFO:tasks.workunit.client.0.smithi167.stderr:++ rbd trash ls 2023-10-05T21:10:45.018 INFO:tasks.workunit.client.0.smithi167.stderr:++ awk '{ print $1 }' 2023-10-05T21:10:45.045 INFO:tasks.workunit.client.0.smithi167.stderr:+ ID=25e37b05efa8 2023-10-05T21:10:45.045 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap purge --image-id 25e37b05efa8 2023-10-05T21:10:45.852 INFO:tasks.workunit.client.0.smithi167.stderr: Removing all snapshots: 100% complete... Removing all snapshots: 100% complete...done. 2023-10-05T21:10:45.864 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash purge 2023-10-05T21:10:45.933 INFO:tasks.workunit.client.0.smithi167.stderr: Removing images: 100% complete... Removing images: 100% complete...done. 2023-10-05T21:10:45.936 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls 2023-10-05T21:10:45.936 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:45.936 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 0 2023-10-05T21:10:45.965 INFO:tasks.workunit.client.0.smithi167.stdout:0 2023-10-05T21:10:45.965 INFO:tasks.workunit.client.0.smithi167.stderr:+ test_deep_copy_clone 2023-10-05T21:10:45.965 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'testing deep copy clone...' 2023-10-05T21:10:45.966 INFO:tasks.workunit.client.0.smithi167.stdout:testing deep copy clone... 2023-10-05T21:10:45.966 INFO:tasks.workunit.client.0.smithi167.stderr:+ remove_images 2023-10-05T21:10:45.966 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:46.050 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:46.132 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:46.214 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:46.296 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:46.378 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:46.457 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:46.540 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:46.628 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:46.709 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:46.792 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:46.876 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:46.959 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:47.043 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:47.327 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:47.411 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:47.494 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:47.781 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:47.862 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create testimg1 --image-format 2 --size 256 2023-10-05T21:10:47.908 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap create testimg1 --snap=snap1 2023-10-05T21:10:48.020 INFO:tasks.workunit.client.0.smithi167.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-05T21:10:48.032 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap protect testimg1@snap1 2023-10-05T21:10:48.074 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd clone testimg1@snap1 testimg2 2023-10-05T21:10:48.138 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap create testimg2@snap2 2023-10-05T21:10:49.019 INFO:tasks.workunit.client.0.smithi167.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-05T21:10:49.028 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd deep copy testimg2 testimg3 2023-10-05T21:10:49.126 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:10:49.125+0000 7f6ca796a700 0 -- 172.21.15.167:0/2439988147 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x55a637b8a810 msgr2=0x55a637c7b1e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:10:50.029 INFO:tasks.workunit.client.0.smithi167.stderr: Image deep copy: 1% complete... Image deep copy: 3% complete... Image deep copy: 4% complete... Image deep copy: 6% complete... Image deep copy: 7% complete... Image deep copy: 9% complete... Image deep copy: 10% complete... Image deep copy: 12% complete... Image deep copy: 14% complete... Image deep copy: 15% complete... Image deep copy: 17% complete... Image deep copy: 18% complete... Image deep copy: 20% complete... Image deep copy: 21% complete... Image deep copy: 23% complete... Image deep copy: 25% complete... Image deep copy: 26% complete... Image deep copy: 28% complete... Image deep copy: 29% complete... Image deep copy: 31% complete... Image deep copy: 32% complete... Image deep copy: 34% complete... Image deep copy: 35% complete... Image deep copy: 37% complete... Image deep copy: 39% complete... Image deep copy: 40% complete... Image deep copy: 42% complete... Image deep copy: 43% complete... Image deep copy: 45% complete... Image deep copy: 46% complete... Image deep copy: 48% complete... Image deep copy: 50% complete... Image deep copy: 51% complete... Image deep copy: 53% complete... Image deep copy: 54% complete... Image deep copy: 56% complete... Image deep copy: 57% complete... Image deep copy: 59% complete... Image deep copy: 60% complete... Image deep copy: 62% complete... Image deep copy: 64% complete... Image deep copy: 65% complete... Image deep copy: 67% complete... Image deep copy: 68% complete... Image deep copy: 70% complete... Image deep copy: 71% complete... Image deep copy: 73% complete... Image deep copy: 75% complete... Image deep copy: 76% complete... Image deep copy: 78% complete... Image deep copy: 79% complete... Image deep copy: 81% complete... Image deep copy: 82% complete... Image deep copy: 84% complete... Image deep copy: 85% complete... Image deep copy: 87% complete... Image deep copy: 89% complete... Image deep copy: 90% complete... Image deep copy: 92% complete... Image deep copy: 93% complete... Image deep copy: 95% complete... Image deep copy: 96% complete... Image deep copy: 98% complete... Image deep copy: 100% complete...2023-10-05T21:10:50.027+0000 7f6ca796a700 0 -- 172.21.15.167:0/2439988147 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x7f6c8c03a600 msgr2=0x7f6c8c03ca20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:10:50.029 INFO:tasks.workunit.client.0.smithi167.stderr: Image deep copy: 100% complete...done. 2023-10-05T21:10:50.033 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info testimg3 2023-10-05T21:10:50.034 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'size 256 MiB' 2023-10-05T21:10:50.077 INFO:tasks.workunit.client.0.smithi167.stdout: size 256 MiB in 64 objects 2023-10-05T21:10:50.077 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info testimg3 2023-10-05T21:10:50.077 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'parent: rbd/testimg1@snap1' 2023-10-05T21:10:50.119 INFO:tasks.workunit.client.0.smithi167.stdout: parent: rbd/testimg1@snap1 2023-10-05T21:10:50.119 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap ls testimg3 2023-10-05T21:10:50.120 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep -v SNAPID 2023-10-05T21:10:50.120 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:50.120 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 1 2023-10-05T21:10:50.161 INFO:tasks.workunit.client.0.smithi167.stdout:1 2023-10-05T21:10:50.162 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap ls testimg3 2023-10-05T21:10:50.162 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '.*snap2.*' 2023-10-05T21:10:50.203 INFO:tasks.workunit.client.0.smithi167.stdout: 76 snap2 256 MiB Thu Oct 5 21:10:50 2023 2023-10-05T21:10:50.203 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info testimg2 2023-10-05T21:10:50.203 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'features:.*deep-flatten' 2023-10-05T21:10:50.245 INFO:tasks.workunit.client.0.smithi167.stdout: features: layering, exclusive-lock, object-map, fast-diff, deep-flatten 2023-10-05T21:10:50.245 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info testimg3 2023-10-05T21:10:50.246 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'features:.*deep-flatten' 2023-10-05T21:10:50.287 INFO:tasks.workunit.client.0.smithi167.stdout: features: layering, exclusive-lock, object-map, fast-diff, deep-flatten 2023-10-05T21:10:50.287 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd flatten testimg2 2023-10-05T21:10:50.336 INFO:tasks.workunit.client.0.smithi167.stderr: Image flatten: 1% complete... Image flatten: 3% complete... Image flatten: 4% complete... Image flatten: 6% complete... Image flatten: 7% complete... Image flatten: 9% complete... Image flatten: 10% complete... Image flatten: 12% complete... Image flatten: 14% complete... Image flatten: 15% complete... Image flatten: 17% complete... Image flatten: 18% complete... Image flatten: 20% complete... Image flatten: 21% complete... Image flatten: 23% complete... Image flatten: 25% complete... Image flatten: 26% complete... Image flatten: 28% complete... Image flatten: 29% complete... Image flatten: 31% complete... Image flatten: 32% complete... Image flatten: 34% complete... Image flatten: 35% complete... Image flatten: 37% complete... Image flatten: 39% complete... Image flatten: 40% complete... Image flatten: 42% complete... Image flatten: 43% complete... Image flatten: 45% complete... Image flatten: 46% complete... Image flatten: 48% complete... Image flatten: 50% complete... Image flatten: 51% complete... Image flatten: 53% complete... Image flatten: 54% complete... Image flatten: 56% complete... Image flatten: 57% complete... Image flatten: 59% complete... Image flatten: 60% complete... Image flatten: 62% complete... Image flatten: 64% complete... Image flatten: 65% complete... Image flatten: 67% complete... Image flatten: 68% complete... Image flatten: 70% complete... Image flatten: 71% complete... Image flatten: 73% complete... Image flatten: 75% complete... Image flatten: 76% complete... Image flatten: 78% complete... Image flatten: 79% complete... Image flatten: 81% complete... Image flatten: 82% complete... Image flatten: 84% complete... Image flatten: 85% complete... Image flatten: 87% complete... Image flatten: 89% complete... Image flatten: 90% complete... Image flatten: 92% complete... Image flatten: 93% complete... Image flatten: 95% complete... Image flatten: 96% complete... Image flatten: 98% complete... Image flatten: 100% complete...done. 2023-10-05T21:10:50.342 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd flatten testimg3 2023-10-05T21:10:50.391 INFO:tasks.workunit.client.0.smithi167.stderr: Image flatten: 1% complete... Image flatten: 3% complete... Image flatten: 4% complete... Image flatten: 6% complete... Image flatten: 7% complete... Image flatten: 9% complete... Image flatten: 10% complete... Image flatten: 12% complete... Image flatten: 14% complete... Image flatten: 15% complete... Image flatten: 17% complete... Image flatten: 18% complete... Image flatten: 20% complete... Image flatten: 21% complete... Image flatten: 23% complete... Image flatten: 25% complete... Image flatten: 26% complete... Image flatten: 28% complete... Image flatten: 29% complete... Image flatten: 31% complete... Image flatten: 32% complete... Image flatten: 34% complete... Image flatten: 35% complete... Image flatten: 37% complete... Image flatten: 39% complete... Image flatten: 40% complete... Image flatten: 42% complete... Image flatten: 43% complete... Image flatten: 45% complete... Image flatten: 46% complete... Image flatten: 48% complete... Image flatten: 50% complete... Image flatten: 51% complete... Image flatten: 53% complete... Image flatten: 54% complete... Image flatten: 56% complete... Image flatten: 57% complete... Image flatten: 59% complete... Image flatten: 60% complete... Image flatten: 62% complete... Image flatten: 64% complete... Image flatten: 65% complete... Image flatten: 67% complete... Image flatten: 68% complete... Image flatten: 70% complete... Image flatten: 71% complete... Image flatten: 73% complete... Image flatten: 75% complete... Image flatten: 76% complete... Image flatten: 78% complete... Image flatten: 79% complete... Image flatten: 81% complete... Image flatten: 82% complete... Image flatten: 84% complete... Image flatten: 85% complete... Image flatten: 87% complete... Image flatten: 89% complete... Image flatten: 90% complete... Image flatten: 92% complete... Image flatten: 93% complete... Image flatten: 95% complete... Image flatten: 96% complete... Image flatten: 98% complete... Image flatten: 100% complete...done. 2023-10-05T21:10:50.398 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap unprotect testimg1@snap1 2023-10-05T21:10:50.442 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap purge testimg2 2023-10-05T21:10:51.022 INFO:tasks.workunit.client.0.smithi167.stderr: Removing all snapshots: 100% complete... Removing all snapshots: 100% complete...done. 2023-10-05T21:10:51.031 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap purge testimg3 2023-10-05T21:10:51.623 INFO:tasks.workunit.client.0.smithi167.stderr: Removing all snapshots: 100% complete... Removing all snapshots: 100% complete...done. 2023-10-05T21:10:51.633 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm testimg2 2023-10-05T21:10:51.745 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 1% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 12% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 23% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 37% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 48% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 62% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 73% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 87% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 98% complete...2023-10-05T21:10:51.743+0000 7f42b60ef700 0 -- 172.21.15.167:0/910801002 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f429c008740 msgr2=0x7f429c00abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:10:51.745 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:10:51.747 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm testimg3 2023-10-05T21:10:51.859 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 1% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 12% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 23% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 37% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 48% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 62% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 73% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 87% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 98% complete...2023-10-05T21:10:51.857+0000 7fc572271700 0 -- 172.21.15.167:0/1059348794 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x565229432f90 msgr2=0x56522940bbe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:10:51.859 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:10:51.862 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap protect testimg1@snap1 2023-10-05T21:10:52.105 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd clone testimg1@snap1 testimg2 2023-10-05T21:10:52.187 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap create testimg2@snap2 2023-10-05T21:10:53.036 INFO:tasks.workunit.client.0.smithi167.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-05T21:10:53.047 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd deep copy --flatten testimg2 testimg3 2023-10-05T21:10:54.166 INFO:tasks.workunit.client.0.smithi167.stderr: Image deep copy: 1% complete... Image deep copy: 3% complete...2023-10-05T21:10:54.164+0000 7fb236052700 0 -- 172.21.15.167:0/1459178767 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7fb2100031a0 msgr2=0x7fb210011c90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:10:54.291 INFO:tasks.workunit.client.0.smithi167.stderr: Image deep copy: 4% complete...2023-10-05T21:10:54.280+0000 7fb236052700 0 -- 172.21.15.167:0/1459178767 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7fb21803aed0 msgr2=0x7fb21803d2b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:10:55.017 INFO:tasks.workunit.client.0.smithi167.stderr: Image deep copy: 6% complete... Image deep copy: 7% complete... Image deep copy: 9% complete... Image deep copy: 10% complete... Image deep copy: 12% complete... Image deep copy: 14% complete... Image deep copy: 15% complete... Image deep copy: 17% complete... Image deep copy: 18% complete... Image deep copy: 20% complete... Image deep copy: 21% complete... Image deep copy: 23% complete... Image deep copy: 25% complete... Image deep copy: 26% complete... Image deep copy: 28% complete... Image deep copy: 29% complete... Image deep copy: 31% complete... Image deep copy: 32% complete... Image deep copy: 34% complete... Image deep copy: 35% complete... Image deep copy: 37% complete... Image deep copy: 39% complete... Image deep copy: 40% complete... Image deep copy: 42% complete... Image deep copy: 43% complete... Image deep copy: 45% complete... Image deep copy: 46% complete... Image deep copy: 48% complete... Image deep copy: 50% complete... Image deep copy: 51% complete... Image deep copy: 53% complete... Image deep copy: 54% complete... Image deep copy: 56% complete... Image deep copy: 57% complete... Image deep copy: 59% complete... Image deep copy: 60% complete... Image deep copy: 62% complete... Image deep copy: 64% complete... Image deep copy: 65% complete... Image deep copy: 67% complete... Image deep copy: 68% complete... Image deep copy: 70% complete... Image deep copy: 71% complete... Image deep copy: 73% complete... Image deep copy: 75% complete... Image deep copy: 76% complete... Image deep copy: 78% complete... Image deep copy: 79% complete... Image deep copy: 81% complete... Image deep copy: 82% complete... Image deep copy: 84% complete... Image deep copy: 85% complete... Image deep copy: 87% complete... Image deep copy: 89% complete... Image deep copy: 90% complete... Image deep copy: 92% complete... Image deep copy: 93% complete... Image deep copy: 95% complete... Image deep copy: 96% complete... Image deep copy: 98% complete... Image deep copy: 100% complete... Image deep copy: 100% complete...done. 2023-10-05T21:10:55.021 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info testimg3 2023-10-05T21:10:55.021 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'size 256 MiB' 2023-10-05T21:10:55.056 INFO:tasks.workunit.client.0.smithi167.stdout: size 256 MiB in 64 objects 2023-10-05T21:10:55.056 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info testimg3 2023-10-05T21:10:55.056 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep -v parent: 2023-10-05T21:10:55.291 INFO:tasks.workunit.client.0.smithi167.stdout:rbd image 'testimg3': 2023-10-05T21:10:55.291 INFO:tasks.workunit.client.0.smithi167.stdout: size 256 MiB in 64 objects 2023-10-05T21:10:55.291 INFO:tasks.workunit.client.0.smithi167.stdout: order 22 (4 MiB objects) 2023-10-05T21:10:55.291 INFO:tasks.workunit.client.0.smithi167.stdout: snapshot_count: 1 2023-10-05T21:10:55.291 INFO:tasks.workunit.client.0.smithi167.stdout: id: 26ef33491117 2023-10-05T21:10:55.291 INFO:tasks.workunit.client.0.smithi167.stdout: block_name_prefix: rbd_data.26ef33491117 2023-10-05T21:10:55.292 INFO:tasks.workunit.client.0.smithi167.stdout: format: 2 2023-10-05T21:10:55.292 INFO:tasks.workunit.client.0.smithi167.stdout: features: layering, exclusive-lock, object-map, fast-diff, deep-flatten 2023-10-05T21:10:55.292 INFO:tasks.workunit.client.0.smithi167.stdout: op_features: 2023-10-05T21:10:55.292 INFO:tasks.workunit.client.0.smithi167.stdout: flags: 2023-10-05T21:10:55.292 INFO:tasks.workunit.client.0.smithi167.stdout: create_timestamp: Thu Oct 5 21:10:53 2023 2023-10-05T21:10:55.292 INFO:tasks.workunit.client.0.smithi167.stdout: access_timestamp: Thu Oct 5 21:10:53 2023 2023-10-05T21:10:55.292 INFO:tasks.workunit.client.0.smithi167.stdout: modify_timestamp: Thu Oct 5 21:10:53 2023 2023-10-05T21:10:55.293 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap ls testimg3 2023-10-05T21:10:55.293 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep -v SNAPID 2023-10-05T21:10:55.293 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:10:55.293 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 1 2023-10-05T21:10:55.325 INFO:tasks.workunit.client.0.smithi167.stdout:1 2023-10-05T21:10:55.326 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap ls testimg3 2023-10-05T21:10:55.326 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '.*snap2.*' 2023-10-05T21:10:55.360 INFO:tasks.workunit.client.0.smithi167.stdout: 80 snap2 256 MiB Thu Oct 5 21:10:54 2023 2023-10-05T21:10:55.360 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd info testimg2 2023-10-05T21:10:55.361 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'features:.*deep-flatten' 2023-10-05T21:10:55.402 INFO:tasks.workunit.client.0.smithi167.stdout: features: layering, exclusive-lock, object-map, fast-diff, deep-flatten 2023-10-05T21:10:55.402 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd flatten testimg2 2023-10-05T21:10:55.456 INFO:tasks.workunit.client.0.smithi167.stderr: Image flatten: 1% complete... Image flatten: 3% complete... Image flatten: 4% complete... Image flatten: 6% complete... Image flatten: 7% complete... Image flatten: 9% complete... Image flatten: 10% complete... Image flatten: 12% complete... Image flatten: 14% complete... Image flatten: 15% complete... Image flatten: 17% complete... Image flatten: 18% complete... Image flatten: 20% complete... Image flatten: 21% complete... Image flatten: 23% complete... Image flatten: 25% complete... Image flatten: 26% complete... Image flatten: 28% complete... Image flatten: 29% complete... Image flatten: 31% complete... Image flatten: 32% complete... Image flatten: 34% complete... Image flatten: 35% complete... Image flatten: 37% complete... Image flatten: 39% complete... Image flatten: 40% complete... Image flatten: 42% complete... Image flatten: 43% complete... Image flatten: 45% complete... Image flatten: 46% complete... Image flatten: 48% complete... Image flatten: 50% complete... Image flatten: 51% complete... Image flatten: 53% complete... Image flatten: 54% complete... Image flatten: 56% complete... Image flatten: 57% complete... Image flatten: 59% complete... Image flatten: 60% complete... Image flatten: 62% complete... Image flatten: 64% complete... Image flatten: 65% complete... Image flatten: 67% complete... Image flatten: 68% complete... Image flatten: 70% complete... Image flatten: 71% complete... Image flatten: 73% complete... Image flatten: 75% complete... Image flatten: 76% complete... Image flatten: 78% complete... Image flatten: 79% complete... Image flatten: 81% complete... Image flatten: 82% complete... Image flatten: 84% complete... Image flatten: 85% complete... Image flatten: 87% complete... Image flatten: 89% complete... Image flatten: 90% complete... Image flatten: 92% complete... Image flatten: 93% complete... Image flatten: 95% complete... Image flatten: 96% complete... Image flatten: 98% complete... Image flatten: 100% complete...done. 2023-10-05T21:10:55.466 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap unprotect testimg1@snap1 2023-10-05T21:10:55.514 INFO:tasks.workunit.client.0.smithi167.stderr:+ remove_images 2023-10-05T21:10:55.514 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:56.165 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:56.758 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:57.840 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:57.924 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:58.009 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:58.090 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:58.173 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:58.255 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:58.345 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:58.429 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:58.512 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:58.596 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:58.679 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:58.760 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:58.843 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:58.927 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:59.018 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:59.100 INFO:tasks.workunit.client.0.smithi167.stderr:+ test_clone_v2 2023-10-05T21:10:59.100 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'testing clone v2...' 2023-10-05T21:10:59.101 INFO:tasks.workunit.client.0.smithi167.stdout:testing clone v2... 2023-10-05T21:10:59.101 INFO:tasks.workunit.client.0.smithi167.stderr:+ remove_images 2023-10-05T21:10:59.101 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:59.187 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:59.269 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:59.355 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:59.438 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:59.520 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:59.602 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:59.685 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:59.769 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:59.850 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:10:59.933 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:00.015 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:00.097 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:00.182 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:00.267 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:00.462 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:00.462 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:00.514 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:00.598 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 2 -s 1 test1 2023-10-05T21:11:00.647 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap create test1@1 2023-10-05T21:11:01.632 INFO:tasks.workunit.client.0.smithi167.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-05T21:11:01.656 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd clone --rbd-default-clone-format=1 test1@1 test2 2023-10-05T21:11:01.684 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:11:01.682+0000 7fc549697700 -1 librbd::image::CloneRequest: 0x562f35adf460 validate_parent: parent snapshot must be protected 2023-10-05T21:11:01.684 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: clone error: (22) Invalid argument 2023-10-05T21:11:01.687 INFO:tasks.workunit.client.0.smithi167.stderr:+ true 2023-10-05T21:11:01.687 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd clone --rbd-default-clone-format=2 test1@1 test2 2023-10-05T21:11:01.756 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd clone --rbd-default-clone-format=2 test1@1 test3 2023-10-05T21:11:01.831 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap protect test1@1 2023-10-05T21:11:01.872 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd clone --rbd-default-clone-format=1 test1@1 test4 2023-10-05T21:11:01.951 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd children test1@1 2023-10-05T21:11:01.951 INFO:tasks.workunit.client.0.smithi167.stderr:+ sort 2023-10-05T21:11:01.952 INFO:tasks.workunit.client.0.smithi167.stderr:+ tr '\n' ' ' 2023-10-05T21:11:01.952 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep -E 'test2.*test3.*test4' 2023-10-05T21:11:01.993 INFO:tasks.workunit.client.0.smithi167.stdout:rbd/test2 rbd/test3 rbd/test4 2023-10-05T21:11:01.994 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd children --descendants test1 2023-10-05T21:11:01.994 INFO:tasks.workunit.client.0.smithi167.stderr:+ sort 2023-10-05T21:11:01.994 INFO:tasks.workunit.client.0.smithi167.stderr:+ tr '\n' ' ' 2023-10-05T21:11:01.994 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep -E 'test2.*test3.*test4' 2023-10-05T21:11:02.048 INFO:tasks.workunit.client.0.smithi167.stdout:rbd/test2 rbd/test3 rbd/test4 2023-10-05T21:11:02.048 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd remove test4 2023-10-05T21:11:02.156 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:11:02.154+0000 7efee8918700 0 -- 172.21.15.167:0/2964163392 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x7efecc03a660 msgr2=0x7efecc03ca40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:11:02.167 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:11:02.170 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap unprotect test1@1 2023-10-05T21:11:02.214 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap remove test1@1 2023-10-05T21:11:02.258 INFO:tasks.workunit.client.0.smithi167.stderr: Removing snap: 100% complete...done. 2023-10-05T21:11:02.267 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap list --all test1 2023-10-05T21:11:02.267 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep -E 'trash \(1\) *$' 2023-10-05T21:11:02.303 INFO:tasks.workunit.client.0.smithi167.stdout: 84 80e98995-be2a-4b0a-886f-553fadec9287 1 MiB Thu Oct 5 21:11:01 2023 trash (1) 2023-10-05T21:11:02.303 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap create test1@2 2023-10-05T21:11:02.634 INFO:tasks.workunit.client.0.smithi167.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-05T21:11:02.645 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test1 2023-10-05T21:11:02.646 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'image has snapshots' 2023-10-05T21:11:02.701 INFO:tasks.workunit.client.0.smithi167.stdout:rbd: image has snapshots - these must be deleted with 'rbd snap purge' before the image can be removed. 2023-10-05T21:11:02.701 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap rm test1@2 2023-10-05T21:11:03.641 INFO:tasks.workunit.client.0.smithi167.stderr: Removing snap: 100% complete...done. 2023-10-05T21:11:03.655 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test1 2023-10-05T21:11:03.655 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'linked clones' 2023-10-05T21:11:03.720 INFO:tasks.workunit.client.0.smithi167.stdout:rbd: image has snapshots with linked clones - these must be deleted or flattened before the image can be removed. 2023-10-05T21:11:03.720 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test3 2023-10-05T21:11:03.823 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:11:03.821+0000 7f2bae5b0700 0 -- 172.21.15.167:0/4259502839 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f2b94008740 msgr2=0x7f2b9400abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:11:03.834 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:11:03.837 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test1 2023-10-05T21:11:03.837 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep 'linked clones' 2023-10-05T21:11:03.894 INFO:tasks.workunit.client.0.smithi167.stdout:rbd: image has snapshots with linked clones - these must be deleted or flattened before the image can be removed. 2023-10-05T21:11:03.894 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd flatten test2 2023-10-05T21:11:04.658 INFO:tasks.workunit.client.0.smithi167.stderr: Image flatten: 100% complete...done. 2023-10-05T21:11:04.669 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap list --all test1 2023-10-05T21:11:04.669 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:11:04.669 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '^0$' 2023-10-05T21:11:04.705 INFO:tasks.workunit.client.0.smithi167.stdout:0 2023-10-05T21:11:04.706 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test1 2023-10-05T21:11:04.818 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:11:04.816+0000 7f24c992d700 0 -- 172.21.15.167:0/4071768332 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f24a8008740 msgr2=0x7f24a800abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:11:04.818 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:11:04.821 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test2 2023-10-05T21:11:04.929 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:11:04.927+0000 7f5af16ee700 0 -- 172.21.15.167:0/787326547 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f5acc008740 msgr2=0x7f5acc00abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:11:04.929 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:11:04.932 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 2 -s 1 test1 2023-10-05T21:11:04.978 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap create test1@1 2023-10-05T21:11:05.656 INFO:tasks.workunit.client.0.smithi167.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-05T21:11:05.666 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap create test1@2 2023-10-05T21:11:06.632 INFO:tasks.workunit.client.0.smithi167.stderr: Creating snap: 10% complete... Creating snap: 100% complete...done. 2023-10-05T21:11:06.643 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd clone test1@1 test2 --rbd-default-clone-format 2 2023-10-05T21:11:06.721 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd clone test1@2 test3 --rbd-default-clone-format 2 2023-10-05T21:11:06.789 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap rm test1@1 2023-10-05T21:11:06.831 INFO:tasks.workunit.client.0.smithi167.stderr: Removing snap: 100% complete...done. 2023-10-05T21:11:06.840 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd snap rm test1@2 2023-10-05T21:11:06.884 INFO:tasks.workunit.client.0.smithi167.stderr: Removing snap: 100% complete...done. 2023-10-05T21:11:06.892 INFO:tasks.workunit.client.0.smithi167.stderr:+ expect_fail rbd rm test1 2023-10-05T21:11:06.893 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test1 2023-10-05T21:11:06.941 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:11:06.939+0000 7f0d757f4540 -1 librbd::api::Image: remove: image has snapshots - not removing 2023-10-05T21:11:06.941 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 0% complete...failed. 2023-10-05T21:11:06.948 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: image has snapshots with linked clones - these must be deleted or flattened before the image can be removed. 2023-10-05T21:11:06.953 INFO:tasks.workunit.client.0.smithi167.stderr:+ return 0 2023-10-05T21:11:06.953 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test1 --rbd-move-parent-to-trash-on-remove=true 2023-10-05T21:11:07.029 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:11:07.032 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls -a 2023-10-05T21:11:07.032 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep test1 2023-10-05T21:11:07.060 INFO:tasks.workunit.client.0.smithi167.stdout:28249c3ed9d6 test1 2023-10-05T21:11:07.060 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test2 2023-10-05T21:11:07.163 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:11:07.161+0000 7f4f0c82c700 0 -- 172.21.15.167:0/2506763814 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f4ef0008740 msgr2=0x7f4ef000abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:11:07.644 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:11:07.642+0000 7f4f0c82c700 0 -- 172.21.15.167:0/2506763814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562aebfc9f60 msgr2=0x562aebfb8440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:11:07.696 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:11:07.700 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls -a 2023-10-05T21:11:07.700 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep test1 2023-10-05T21:11:07.730 INFO:tasks.workunit.client.0.smithi167.stdout:28249c3ed9d6 test1 2023-10-05T21:11:07.730 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test3 2023-10-05T21:11:07.836 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:11:07.835+0000 7f0eb35c0700 0 -- 172.21.15.167:0/536718289 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x56044bb12240 msgr2=0x56044bb38b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:11:08.646 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:11:08.644+0000 7f0eb35c0700 0 -- 172.21.15.167:0/536718289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56044ba22f60 msgr2=0x56044bab39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:11:08.716 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:11:08.719 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd trash ls -a 2023-10-05T21:11:08.720 INFO:tasks.workunit.client.0.smithi167.stderr:+ expect_fail grep test1 2023-10-05T21:11:08.720 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep test1 2023-10-05T21:11:08.749 INFO:tasks.workunit.client.0.smithi167.stderr:+ return 0 2023-10-05T21:11:08.749 INFO:tasks.workunit.client.0.smithi167.stderr:+ test_thick_provision 2023-10-05T21:11:08.749 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'testing thick provision...' 2023-10-05T21:11:08.749 INFO:tasks.workunit.client.0.smithi167.stdout:testing thick provision... 2023-10-05T21:11:08.750 INFO:tasks.workunit.client.0.smithi167.stderr:+ remove_images 2023-10-05T21:11:08.750 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:08.833 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:08.915 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:08.997 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:09.083 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:09.166 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:09.450 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:09.534 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:09.617 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:09.702 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:09.784 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:09.873 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:09.955 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:10.037 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:10.121 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:10.204 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:10.288 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:10.373 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:10.455 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 2 --thick-provision -s 64M test1 2023-10-05T21:11:10.878 INFO:tasks.workunit.client.0.smithi167.stderr: Thick provisioning: 6% complete... Thick provisioning: 12% complete... Thick provisioning: 18% complete... Thick provisioning: 25% complete... Thick provisioning: 31% complete... Thick provisioning: 37% complete... Thick provisioning: 43% complete... Thick provisioning: 50% complete... Thick provisioning: 56% complete... Thick provisioning: 62% complete... Thick provisioning: 68% complete... Thick provisioning: 75% complete... Thick provisioning: 81% complete... Thick provisioning: 87% complete... Thick provisioning: 93% complete... Thick provisioning: 100% complete... Thick provisioning: 100% complete...done. 2023-10-05T21:11:10.891 INFO:tasks.workunit.client.0.smithi167.stderr:+ count=0 2023-10-05T21:11:10.891 INFO:tasks.workunit.client.0.smithi167.stderr:+ ret= 2023-10-05T21:11:10.892 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 0 -lt 10 ']' 2023-10-05T21:11:10.892 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd du 2023-10-05T21:11:10.892 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep test1 2023-10-05T21:11:10.892 INFO:tasks.workunit.client.0.smithi167.stderr:+ tr -s ' ' 2023-10-05T21:11:10.892 INFO:tasks.workunit.client.0.smithi167.stderr:+ cut -d ' ' -f 4-5 2023-10-05T21:11:10.892 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '^64 MiB' 2023-10-05T21:11:10.925 INFO:tasks.workunit.client.0.smithi167.stdout:64 MiB 2023-10-05T21:11:10.925 INFO:tasks.workunit.client.0.smithi167.stderr:+ ret=0 2023-10-05T21:11:10.925 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 0 = 0 ']' 2023-10-05T21:11:10.925 INFO:tasks.workunit.client.0.smithi167.stderr:+ break 2023-10-05T21:11:10.926 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd du 2023-10-05T21:11:10.956 INFO:tasks.workunit.client.0.smithi167.stdout:NAME PROVISIONED USED 2023-10-05T21:11:10.956 INFO:tasks.workunit.client.0.smithi167.stdout:test1 64 MiB 64 MiB 2023-10-05T21:11:10.959 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 0 '!=' 0 ']' 2023-10-05T21:11:10.959 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test1 2023-10-05T21:11:11.066 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 6% complete... Removing image: 12% complete... Removing image: 18% complete... Removing image: 25% complete... Removing image: 31% complete... Removing image: 37% complete... Removing image: 43% complete... Removing image: 50% complete... Removing image: 56% complete... Removing image: 62% complete... Removing image: 68% complete... Removing image: 75% complete... Removing image: 81% complete... Removing image: 87% complete... Removing image: 93% complete...2023-10-05T21:11:11.066+0000 7fd44546a700 0 -- 172.21.15.167:0/3651475727 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x5576ec442320 msgr2=0x5576ec456c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:11:11.110 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:11:11.109+0000 7fd444c69700 0 -- 172.21.15.167:0/3651475727 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7fd428008740 msgr2=0x7fd42800abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:11:11.111 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 100% complete...done. 2023-10-05T21:11:11.114 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls 2023-10-05T21:11:11.114 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep test1 2023-10-05T21:11:11.114 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:11:11.114 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '^0$' 2023-10-05T21:11:11.142 INFO:tasks.workunit.client.0.smithi167.stdout:0 2023-10-05T21:11:11.142 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 2 --thick-provision -s 4G test1 2023-10-05T21:11:11.702 INFO:tasks.workunit.client.0.smithi167.stderr: Thick provisioning: 1% complete... Thick provisioning: 2% complete... Thick provisioning: 3% complete... Thick provisioning: 4% complete... Thick provisioning: 5% complete...2023-10-05T21:11:11.701+0000 7f4b93054700 0 -- 172.21.15.167:0/374064904 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x560692e89cf0 msgr2=0x560692f288d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:11:11.848 INFO:tasks.workunit.client.0.smithi167.stderr: Thick provisioning: 6% complete...2023-10-05T21:11:11.848+0000 7f4b92853700 0 -- 172.21.15.167:0/374064904 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x560692f7a810 msgr2=0x560692f7cc90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:11:16.000 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:11:15.999+0000 7f6cb4555700 -1 reset not still connected to 0x55ddaa044c30 2023-10-05T21:11:22.119 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:11:22.116+0000 7f6cb4555700 -1 reset not still connected to 0x55ddaa044c30 2023-10-05T21:11:25.343 INFO:tasks.workunit.client.0.smithi167.stderr: Thick provisioning: 7% complete... Thick provisioning: 8% complete... Thick provisioning: 9% complete... Thick provisioning: 10% complete... Thick provisioning: 11% complete... Thick provisioning: 12% complete... Thick provisioning: 13% complete... Thick provisioning: 14% complete... Thick provisioning: 15% complete... Thick provisioning: 16% complete... Thick provisioning: 17% complete... Thick provisioning: 18% complete... Thick provisioning: 19% complete... Thick provisioning: 20% complete... Thick provisioning: 21% complete... Thick provisioning: 22% complete... Thick provisioning: 23% complete... Thick provisioning: 24% complete... Thick provisioning: 25% complete... Thick provisioning: 26% complete... Thick provisioning: 27% complete... Thick provisioning: 28% complete... Thick provisioning: 29% complete... Thick provisioning: 30% complete... Thick provisioning: 31% complete... Thick provisioning: 32% complete... Thick provisioning: 33% complete... Thick provisioning: 34% complete... Thick provisioning: 35% complete... Thick provisioning: 36% complete... Thick provisioning: 37% complete... Thick provisioning: 38% complete... Thick provisioning: 39% complete... Thick provisioning: 40% complete... Thick provisioning: 41% complete... Thick provisioning: 42% complete... Thick provisioning: 43% complete... Thick provisioning: 44% complete... Thick provisioning: 45% complete... Thick provisioning: 46% complete... Thick provisioning: 47% complete... Thick provisioning: 48% complete... Thick provisioning: 49% complete... Thick provisioning: 50% complete... Thick provisioning: 51% complete... Thick provisioning: 52% complete... Thick provisioning: 53% complete... Thick provisioning: 54% complete... Thick provisioning: 55% complete... Thick provisioning: 56% complete... Thick provisioning: 57% complete... Thick provisioning: 58% complete... Thick provisioning: 59% complete... Thick provisioning: 60% complete... Thick provisioning: 61% complete... Thick provisioning: 62% complete... Thick provisioning: 63% complete... Thick provisioning: 64% complete... Thick provisioning: 65% complete... Thick provisioning: 66% complete... Thick provisioning: 67% complete... Thick provisioning: 68% complete... Thick provisioning: 69% complete... Thick provisioning: 70% complete... Thick provisioning: 71% complete... Thick provisioning: 72% complete... Thick provisioning: 73% complete... Thick provisioning: 74% complete... Thick provisioning: 75% complete... Thick provisioning: 76% complete... Thick provisioning: 77% complete... Thick provisioning: 78% complete... Thick provisioning: 79% complete... Thick provisioning: 80% complete... Thick provisioning: 81% complete... Thick provisioning: 82% complete... Thick provisioning: 83% complete... Thick provisioning: 84% complete... Thick provisioning: 85% complete... Thick provisioning: 86% complete... Thick provisioning: 87% complete... Thick provisioning: 88% complete... Thick provisioning: 89% complete... Thick provisioning: 90% complete... Thick provisioning: 91% complete... Thick provisioning: 92% complete... Thick provisioning: 93% complete... Thick provisioning: 94% complete... Thick provisioning: 95% complete... Thick provisioning: 96% complete... Thick provisioning: 97% complete... Thick provisioning: 98% complete... Thick provisioning: 99% complete... Thick provisioning: 100% complete... Thick provisioning: 100% complete...done. 2023-10-05T21:11:25.354 INFO:tasks.workunit.client.0.smithi167.stderr:+ count=0 2023-10-05T21:11:25.355 INFO:tasks.workunit.client.0.smithi167.stderr:+ ret= 2023-10-05T21:11:25.355 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 0 -lt 10 ']' 2023-10-05T21:11:25.355 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd du 2023-10-05T21:11:25.355 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep test1 2023-10-05T21:11:25.355 INFO:tasks.workunit.client.0.smithi167.stderr:+ tr -s ' ' 2023-10-05T21:11:25.355 INFO:tasks.workunit.client.0.smithi167.stderr:+ cut -d ' ' -f 4-5 2023-10-05T21:11:25.355 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '^4 GiB' 2023-10-05T21:11:25.389 INFO:tasks.workunit.client.0.smithi167.stdout:4 GiB 2023-10-05T21:11:25.389 INFO:tasks.workunit.client.0.smithi167.stderr:+ ret=0 2023-10-05T21:11:25.390 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 0 = 0 ']' 2023-10-05T21:11:25.390 INFO:tasks.workunit.client.0.smithi167.stderr:+ break 2023-10-05T21:11:25.390 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd du 2023-10-05T21:11:25.422 INFO:tasks.workunit.client.0.smithi167.stdout:NAME PROVISIONED USED 2023-10-05T21:11:25.422 INFO:tasks.workunit.client.0.smithi167.stdout:test1 4 GiB 4 GiB 2023-10-05T21:11:25.424 INFO:tasks.workunit.client.0.smithi167.stderr:+ '[' 0 '!=' 0 ']' 2023-10-05T21:11:25.424 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd rm test1 2023-10-05T21:11:25.540 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 1% complete...2023-10-05T21:11:25.539+0000 7f508f491700 0 -- 172.21.15.167:0/495809323 >> [v2:172.21.15.167:6808/144115,v1:172.21.15.167:6809/144115] conn(0x565159698870 msgr2=0x56515973cf20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:11:25.557 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 2% complete... Removing image: 3% complete...2023-10-05T21:11:25.556+0000 7f508ec90700 0 -- 172.21.15.167:0/495809323 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x5651597d0020 msgr2=0x5651597bb5d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:11:26.434 INFO:tasks.workunit.client.0.smithi167.stderr: Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2023-10-05T21:11:26.438 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd ls 2023-10-05T21:11:26.438 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep test1 2023-10-05T21:11:26.438 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:11:26.438 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '^0$' 2023-10-05T21:11:26.466 INFO:tasks.workunit.client.0.smithi167.stdout:0 2023-10-05T21:11:26.466 INFO:tasks.workunit.client.0.smithi167.stderr:+ test_namespace 2023-10-05T21:11:26.466 INFO:tasks.workunit.client.0.smithi167.stderr:+ echo 'testing namespace...' 2023-10-05T21:11:26.467 INFO:tasks.workunit.client.0.smithi167.stdout:testing namespace... 2023-10-05T21:11:26.467 INFO:tasks.workunit.client.0.smithi167.stderr:+ remove_images 2023-10-05T21:11:26.467 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:26.551 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:26.636 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:26.720 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:26.809 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:26.894 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:26.977 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:27.061 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:27.146 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:27.236 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:27.321 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:27.406 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:27.489 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:27.573 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:27.659 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:27.742 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:27.827 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:27.910 INFO:tasks.workunit.client.0.smithi167.stderr:+ for img in $IMGS 2023-10-05T21:11:27.996 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd namespace ls 2023-10-05T21:11:27.996 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:11:27.996 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '^0$' 2023-10-05T21:11:28.024 INFO:tasks.workunit.client.0.smithi167.stdout:0 2023-10-05T21:11:28.024 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd namespace create rbd/test1 2023-10-05T21:11:28.058 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd namespace create --pool rbd --namespace test2 2023-10-05T21:11:28.093 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd namespace create --namespace test3 2023-10-05T21:11:28.128 INFO:tasks.workunit.client.0.smithi167.stderr:+ expect_fail rbd namespace create rbd/test3 2023-10-05T21:11:28.128 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd namespace create rbd/test3 2023-10-05T21:11:28.157 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: failed to created namespace: (17) File exists 2023-10-05T21:11:28.157 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:11:28.156+0000 7f4465b46540 -1 librbd::api::Namespace: create: failed to add namespace: (17) File exists 2023-10-05T21:11:28.160 INFO:tasks.workunit.client.0.smithi167.stderr:+ return 0 2023-10-05T21:11:28.160 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd namespace list 2023-10-05T21:11:28.160 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep test 2023-10-05T21:11:28.160 INFO:tasks.workunit.client.0.smithi167.stderr:+ wc -l 2023-10-05T21:11:28.160 INFO:tasks.workunit.client.0.smithi167.stderr:+ grep '^3$' 2023-10-05T21:11:28.187 INFO:tasks.workunit.client.0.smithi167.stdout:3 2023-10-05T21:11:28.188 INFO:tasks.workunit.client.0.smithi167.stderr:+ expect_fail rbd namespace remove --pool rbd missing 2023-10-05T21:11:28.188 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd namespace remove --pool rbd missing 2023-10-05T21:11:28.210 INFO:tasks.workunit.client.0.smithi167.stderr:rbd: namespace name was not specified 2023-10-05T21:11:28.211 INFO:tasks.workunit.client.0.smithi167.stderr:+ return 0 2023-10-05T21:11:28.211 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd create --image-format 2 --size 1G rbd/test1/image1 2023-10-05T21:11:28.257 INFO:tasks.workunit.client.0.smithi167.stderr:+ rbd bench --io-type write --io-pattern rand --io-total 32M --io-size 4K rbd/test1/image1 2023-10-05T21:11:28.315 INFO:tasks.workunit.client.0.smithi167.stdout:bench type write io_size 4096 io_threads 16 bytes 33554432 pattern random 2023-10-05T21:11:28.702 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T21:11:28.699+0000 7f994396e700 0 -- 172.21.15.167:0/1156974075 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x55b130ed3000 msgr2=0x55b130eeb3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2023-10-05T21:11:58.740 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:11:58.738+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 161 slow ops, oldest is osd_op(client.10585.0:347 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3231744~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:11:59.032 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:11:59.030+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.10585.0:320 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 233472~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:11:59.711 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:11:59.709+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 161 slow ops, oldest is osd_op(client.10585.0:347 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3231744~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:00.005 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:00.003+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.10585.0:320 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 233472~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:00.699 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:00.697+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 161 slow ops, oldest is osd_op(client.10585.0:347 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3231744~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:00.986 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:00.984+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.10585.0:320 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 233472~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:01.746 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:01.744+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:02.032 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:02.030+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:02.759 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:02.757+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:03.039 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:03.005+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:03.806 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:03.804+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:04.058 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:04.056+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:04.809 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:04.807+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:05.075 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:05.074+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:05.786 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:05.785+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:06.117 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:06.116+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:06.782 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:06.781+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:07.071 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:07.070+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:07.738 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:07.737+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:08.037 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:08.036+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:08.746 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:08.745+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:09.075 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:09.074+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:09.750 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:09.749+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:10.059 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:10.058+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:10.781 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:10.780+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:11.094 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:11.093+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:11.758 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:11.757+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:12.105 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:12.104+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:12.744 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:12.743+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:13.108 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:13.107+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:13.757 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:13.757+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:14.148 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:14.147+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:14.796 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:14.795+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:15.197 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:15.196+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:15.830 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:15.828+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:16.237 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:16.236+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:16.786 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:16.785+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:17.196 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:17.195+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:17.834 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:17.833+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:18.219 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:18.218+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:18.836 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:18.835+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:19.221 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:19.220+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:19.837 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:19.836+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:20.266 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:20.265+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:20.884 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:20.883+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:21.290 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:21.289+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:21.850 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:21.849+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:22.333 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:22.332+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:22.881 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:22.880+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:23.374 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:23.373+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:23.840 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:23.839+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:24.334 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:24.333+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:24.879 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:24.879+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:25.341 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:25.340+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:25.913 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:25.908+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:26.359 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:26.357+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:26.876 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:26.874+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:27.370 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:27.367+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:27.873 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:27.871+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:28.390 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:28.388+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:28.889 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:28.887+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:29.416 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:29.414+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:29.918 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:29.916+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:30.433 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:30.431+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:30.915 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:30.913+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:31.412 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:31.410+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:31.891 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:31.889+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:32.374 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:32.372+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:32.909 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:32.906+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:33.395 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:33.393+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:33.924 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:33.922+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:34.369 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:34.367+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:34.936 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:34.934+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:35.341 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:35.339+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:35.898 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:35.896+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:36.300 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:36.298+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:36.898 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:36.896+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:37.350 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:37.348+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:37.852 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:37.850+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:38.362 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:38.360+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:38.836 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:38.834+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:39.330 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:39.328+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:39.820 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:39.818+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:40.314 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:40.312+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:40.855 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:40.853+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:41.294 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:41.292+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:41.887 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:41.885+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:42.278 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:42.276+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:42.856 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:42.854+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:43.273 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:43.271+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:43.808 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:43.806+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:44.292 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:44.290+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:44.792 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:44.790+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:45.285 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:45.283+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:45.830 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:45.828+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:46.302 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:46.300+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:46.845 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:46.843+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:47.267 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:47.265+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:47.807 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:47.805+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:48.242 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:48.241+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:48.856 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:48.854+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:49.290 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:49.288+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:49.844 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:49.842+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:50.243 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:50.241+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:50.805 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:50.803+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:51.254 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:51.252+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:51.818 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:51.816+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:52.240 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:52.238+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:52.806 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:52.804+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:53.258 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:53.256+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:53.835 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:53.833+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:54.271 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:54.269+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:54.873 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:54.871+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:55.242 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:55.240+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:55.854 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:55.852+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:58.844 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:56.242+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:58.850 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:56.895+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:58.850 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:57.241+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:58.851 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:57.855+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:58.851 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:58.206+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:58.879 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:58.877+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:59.554 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:12:59.224+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:12:59.830 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:12:59.828+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:00.187 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:00.185+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:00.808 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:00.806+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:01.200 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:01.198+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:01.817 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:01.815+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:02.231 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:02.229+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:02.770 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:02.768+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:03.186 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:03.185+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:03.751 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:03.750+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:04.213 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:04.212+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:04.783 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:04.782+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:05.174 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:05.173+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:05.828 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:05.827+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:06.540 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:06.155+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:06.807 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:06.806+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:07.120 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:07.119+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:07.792 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:07.791+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:08.153 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:08.152+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:08.827 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:08.827+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:09.148 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:09.147+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:09.858 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:09.857+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:10.117 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:10.116+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:10.851 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:10.850+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:11.072 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:11.071+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:11.848 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:11.847+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:12.088 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:12.087+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:12.842 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:12.841+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:13.109 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:13.108+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:13.874 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:13.873+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:14.121 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:14.120+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:14.836 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:14.835+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:15.158 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:15.157+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:15.830 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:15.829+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:16.142 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:16.141+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:16.839 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:16.838+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:17.114 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:17.113+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:17.836 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:17.835+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:18.075 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:18.074+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:18.790 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:18.789+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:19.086 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:19.085+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:19.780 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:19.779+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:20.041 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:20.039+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:20.733 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:20.732+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:21.085 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:21.084+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:21.761 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:21.760+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:22.068 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:22.067+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:22.805 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:22.804+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:23.087 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:23.085+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:23.760 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:23.758+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:24.104 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:24.102+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:24.770 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:24.768+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:25.114 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:25.111+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:25.820 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:25.818+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:26.074 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:26.072+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:26.788 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:26.786+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:27.097 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:27.095+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:27.787 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:27.785+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:28.094 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:28.092+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:28.755 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:28.752+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:29.086 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:29.084+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:29.758 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:29.755+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:30.046 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:30.044+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:30.728 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:30.726+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:31.068 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:31.066+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:31.730 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:31.728+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:32.021 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:32.019+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:32.715 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:32.713+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:32.982 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:32.980+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:33.686 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:33.684+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:33.977 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:33.975+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:34.693 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:34.691+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:34.952 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:34.950+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:35.722 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:35.720+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:35.921 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:35.918+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:36.763 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:36.761+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:36.969 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:36.967+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:37.715 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:37.713+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:37.928 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:37.926+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:38.756 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:38.754+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:38.926 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:38.924+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:39.791 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:39.789+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:39.927 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:39.925+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:40.838 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:40.836+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:40.946 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:40.944+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:41.851 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:41.849+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:41.952 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:41.950+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:42.802 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:42.800+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:42.999 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:42.997+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:43.838 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:43.836+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:43.962 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:43.960+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:44.834 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:44.832+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:44.966 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:44.963+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:45.843 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:45.841+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:46.015 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:46.013+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:46.858 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:46.856+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:47.042 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:47.040+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:47.817 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:47.815+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:48.044 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:48.042+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:48.777 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:48.775+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:49.054 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:49.052+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:49.778 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:49.776+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:50.024 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:50.021+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:50.796 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:50.794+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:51.073 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:51.071+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:51.754 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:51.752+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:52.095 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:52.094+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:52.728 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:52.726+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:53.076 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:53.074+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:53.684 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:53.682+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:54.045 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:54.043+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:54.731 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:54.729+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:55.039 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:55.037+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:55.713 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:55.711+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:56.063 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:56.061+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:56.709 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:56.706+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:57.104 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:57.102+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:57.745 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:57.743+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:58.132 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:58.130+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:58.769 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:58.767+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:59.111 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:13:59.109+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:13:59.774 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:13:59.772+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:00.140 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:00.138+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:00.796 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:00.795+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:06.018 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:01.139+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:06.018 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:01.833+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:06.024 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:02.159+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:06.025 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:02.802+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:06.028 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:03.140+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:06.029 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:03.781+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:06.029 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:04.124+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:06.030 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:04.737+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:06.030 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:05.147+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:06.031 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:05.766+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:06.130 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:06.129+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:06.759 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:06.758+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:07.169 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:07.168+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:07.726 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:07.725+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:08.183 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:08.182+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:08.754 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:08.753+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:09.206 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:09.205+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:09.724 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:09.723+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:10.231 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:10.230+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:10.686 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:10.685+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:11.188 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:11.187+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:11.676 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:11.675+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:12.222 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:12.220+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:12.718 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:12.716+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:13.252 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:13.251+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:13.752 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:13.751+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:14.294 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:14.293+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:14.733 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:14.732+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:15.324 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:15.323+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:15.690 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:15.689+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:16.319 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:16.318+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:16.719 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:16.717+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:17.344 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:17.343+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:17.730 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:17.729+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:18.344 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:18.343+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:18.714 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:18.713+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:19.318 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:19.317+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:19.726 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:19.725+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:20.336 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:20.335+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:20.760 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:20.759+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:21.380 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:21.378+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:21.742 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:21.740+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:22.420 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:22.418+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:22.704 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:22.702+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:23.436 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:23.434+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:23.744 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:23.742+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:24.402 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:24.400+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:25.406 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:24.729+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:25.406 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:25.372+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:25.760 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:25.758+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:26.350 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:26.348+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:26.762 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:26.760+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:27.382 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:27.380+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:27.736 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:27.734+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:28.424 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:28.422+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:28.740 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:28.738+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:29.433 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:29.431+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:29.900 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:29.781+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:30.448 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:30.446+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:31.072 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:30.823+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:31.471 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:31.469+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:31.788 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:31.786+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:32.481 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:32.478+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:32.759 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:32.757+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:33.466 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:33.464+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:33.781 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:33.779+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:34.435 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:34.433+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:34.761 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:34.759+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:35.415 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:35.413+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:35.771 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:35.769+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:36.449 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:36.447+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:36.742 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:36.740+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:37.458 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:37.455+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:37.754 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:37.752+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:38.483 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:38.480+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:38.794 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:38.792+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:39.460 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:39.458+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:39.759 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:39.757+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:40.501 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:40.498+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:40.784 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:40.782+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:41.539 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:41.537+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:41.745 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:41.743+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:42.537 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:42.535+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:42.773 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:42.771+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:43.513 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:43.510+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:43.789 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:43.787+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:44.480 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:44.478+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:44.825 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:44.822+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:45.442 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:45.440+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:45.801 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:45.799+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:46.420 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:46.418+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:46.840 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:46.838+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:47.454 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:47.452+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:47.872 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:47.870+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:48.412 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:48.410+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:48.868 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:48.866+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:49.384 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:49.382+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:49.877 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:49.875+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:50.415 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:50.412+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:50.859 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:50.857+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:51.389 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:51.387+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:51.862 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:51.860+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:52.409 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:52.407+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:52.912 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:52.910+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:53.370 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:53.368+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:53.949 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:53.947+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:54.356 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:54.354+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:54.936 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:54.934+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:55.326 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:55.324+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:55.891 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:55.889+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:56.350 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:56.348+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:56.842 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:56.840+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:57.312 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:57.310+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:57.800 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:57.798+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:58.284 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:58.282+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:58.762 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:58.761+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:59.238 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:14:59.237+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:14:59.724 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:14:59.723+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:00.281 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:00.280+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:00.723 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:00.722+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:01.233 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:01.232+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:01.771 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:01.770+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:02.187 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:02.186+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:02.747 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:02.746+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:03.167 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:03.166+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:03.784 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:03.783+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:04.210 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:04.210+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:04.752 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:04.751+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:05.201 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:05.200+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:05.727 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:05.726+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:06.234 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:06.233+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:06.722 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:06.721+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:07.248 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:07.247+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:07.693 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:07.692+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:08.289 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:08.288+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:08.688 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:08.687+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:09.277 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:09.276+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:09.718 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:09.717+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:10.231 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:10.230+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:10.671 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:10.670+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:11.221 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:11.220+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:11.702 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:11.701+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:12.216 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:12.215+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:12.739 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:12.738+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:13.261 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:13.260+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:13.779 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:13.778+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:14.279 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:14.278+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:14.819 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:14.818+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:15.253 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:15.252+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:15.861 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:15.860+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:16.292 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:16.291+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:16.848 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:16.847+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:17.261 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:17.260+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:17.864 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:17.863+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:18.291 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:18.290+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:18.843 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:18.842+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:19.264 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:19.262+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:19.875 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:19.873+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:20.237 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:20.235+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:20.911 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:20.909+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:21.271 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:21.268+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:21.913 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:21.911+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:22.266 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:22.264+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:22.881 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:22.879+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:23.255 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:23.253+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:23.930 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:23.928+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:24.283 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:24.281+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:24.900 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:24.898+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:25.293 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:25.291+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:25.897 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:25.895+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:26.281 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:26.279+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:26.935 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:26.933+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:27.301 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:27.299+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:27.957 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:27.955+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:28.264 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:28.262+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:28.961 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:28.959+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:29.303 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:29.301+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:29.927 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:29.925+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:30.279 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:30.277+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:30.886 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:30.884+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:31.301 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:31.299+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:31.863 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:31.861+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:32.263 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:32.261+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:32.885 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:32.883+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:33.270 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:33.268+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:33.876 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:33.874+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:34.262 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:34.260+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:34.852 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:34.850+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:35.313 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:35.311+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:35.805 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:35.803+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:36.315 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:36.313+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:36.767 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:36.765+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:37.292 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:37.290+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:37.721 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:37.719+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:38.265 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:38.263+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:38.731 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:38.729+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:39.285 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:39.283+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:39.689 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:39.687+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:40.296 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:40.294+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:40.689 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:40.687+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:41.268 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:41.266+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:41.723 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:41.720+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:42.234 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:42.232+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:42.771 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:42.769+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:43.239 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:43.237+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:43.752 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:43.750+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:44.237 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:44.235+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:44.757 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:44.755+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:45.261 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:45.259+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:45.765 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:45.763+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:46.228 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:46.226+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:46.743 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:46.741+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:47.253 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:47.251+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:47.719 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:47.717+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:48.251 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:48.249+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:48.727 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:48.725+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:49.210 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:49.208+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:49.749 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:49.747+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:50.239 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:50.237+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:50.796 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:50.794+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:51.231 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:51.229+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:51.779 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:51.777+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:52.185 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:52.183+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:52.765 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:52.763+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:53.136 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:53.134+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:53.749 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:53.747+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:54.150 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:54.148+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:54.751 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:54.749+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:55.117 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:55.115+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:55.710 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:55.708+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:56.094 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:56.092+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:56.760 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:56.759+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:57.074 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:57.073+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:57.742 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:57.741+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:58.025 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:58.024+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:58.705 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:58.704+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:58.988 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:58.987+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:59.666 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:15:59.664+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:15:59.978 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:15:59.977+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:00.633 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:00.632+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:00.966 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:00.965+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:01.659 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:01.658+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:01.971 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:01.970+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:02.664 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:02.663+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:02.961 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:02.960+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:03.650 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:03.648+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:03.969 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:03.968+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:04.692 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:04.691+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:04.976 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:04.975+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:05.694 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:05.693+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:06.002 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:06.001+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:06.691 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:06.690+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:06.979 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:06.978+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:07.644 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:07.643+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:08.026 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:08.025+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:08.609 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:08.608+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:09.052 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:09.051+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:09.613 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:09.612+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:10.086 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:10.085+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:10.652 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:10.651+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:11.064 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:11.063+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:11.668 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:11.667+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:12.062 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:12.061+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:12.705 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:12.704+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:13.088 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:13.087+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:13.721 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:13.720+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:14.081 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:14.080+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:14.700 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:14.699+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:15.060 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:15.059+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:15.728 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:15.726+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:16.036 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:16.035+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:16.700 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:16.699+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:17.071 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:17.069+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:17.692 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:17.690+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:18.052 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:18.049+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:18.685 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:18.682+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:19.015 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:19.013+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:19.726 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:19.724+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:20.035 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:20.033+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:20.738 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:20.736+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:21.010 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:21.008+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:21.779 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:21.777+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:22.002 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:22.000+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:22.758 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:22.756+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:23.008 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:23.006+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:23.765 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:23.763+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:23.959 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:23.958+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:24.794 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:24.792+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:24.985 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:24.984+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:25.783 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:25.781+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:26.000 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:25.998+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:26.752 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:26.750+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:27.005 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:27.003+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:27.770 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:27.768+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:28.028 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:28.026+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:28.759 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:28.757+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:29.006 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:29.003+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:29.726 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:29.724+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:30.041 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:30.039+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:30.750 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:30.747+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:31.047 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:31.044+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:31.798 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:31.796+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:32.050 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:32.048+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:32.825 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:32.823+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:33.079 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:33.077+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:33.863 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:33.861+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:34.072 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:34.070+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:34.895 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:34.893+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:35.074 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:35.072+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:35.913 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:35.911+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:36.037 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:36.035+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:36.884 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:36.881+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:36.997 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:36.996+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:37.840 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:37.838+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:38.000 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:37.998+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:38.808 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:38.806+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:39.041 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:39.039+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:39.832 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:39.830+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:40.003 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:40.001+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:40.802 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:40.800+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:40.998 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:40.996+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:41.756 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:41.754+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:41.989 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:41.987+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:42.720 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:42.718+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:42.998 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:42.996+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:43.757 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:43.755+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:43.951 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:43.949+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:44.724 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:44.722+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:44.962 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:44.960+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:45.679 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:45.677+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:45.978 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:45.976+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:46.694 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:46.692+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:47.019 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:47.017+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:47.658 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:47.655+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:48.026 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:48.024+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:48.639 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:48.636+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:48.987 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:48.985+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:49.616 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:49.614+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:49.958 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:49.956+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:50.584 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:50.582+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:50.972 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:50.970+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:51.548 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:51.546+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:52.002 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:52.000+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:52.585 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:52.583+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:53.029 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:53.027+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:53.634 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:53.631+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:54.038 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:54.036+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:54.661 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:54.659+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:55.054 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:55.053+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:55.659 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:55.658+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:56.091 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:56.088+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:56.617 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:56.616+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:57.097 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:57.096+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:57.596 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:57.595+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:58.112 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:58.111+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:58.575 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:58.574+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:59.161 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:16:59.160+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:16:59.623 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:16:59.622+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:00.203 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:00.202+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:00.590 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:00.589+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:01.224 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:01.223+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:01.587 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:01.586+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:02.183 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:02.182+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:02.567 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:02.566+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:03.222 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:03.220+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:03.609 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:03.608+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:04.230 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:04.229+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:04.595 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:04.594+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:05.279 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:05.277+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:05.600 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:05.599+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:06.269 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:06.268+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:06.572 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:06.571+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:07.228 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:07.227+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:07.605 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:07.603+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:08.216 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:08.215+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:08.585 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:08.584+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:09.260 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:09.259+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:09.623 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:09.622+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:10.283 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:10.282+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:10.670 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:10.669+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:11.254 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:11.253+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:11.680 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:11.679+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:12.232 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:12.231+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:12.705 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:12.703+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:13.203 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:13.202+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:13.751 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:13.750+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:14.210 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:14.209+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:14.774 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:14.773+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:15.176 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:15.175+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:15.754 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:15.752+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:16.135 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:16.133+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:16.796 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:16.794+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:17.167 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:17.165+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:17.790 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:17.788+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:18.218 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:18.216+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:18.777 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:18.775+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:19.217 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:19.215+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:19.759 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:19.757+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:20.190 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:20.188+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:20.784 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:20.782+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:21.225 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:21.223+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:21.819 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:21.817+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:22.205 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:22.203+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:22.795 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:22.793+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:23.222 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:23.220+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:23.841 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:23.839+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:24.193 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:24.190+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:24.843 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:24.841+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:25.179 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:25.177+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:25.821 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:25.819+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:26.153 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:26.150+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:26.799 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:26.797+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:27.197 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:27.195+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:27.750 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:27.748+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:28.217 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:28.215+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:28.749 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:28.747+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:29.168 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:29.166+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:29.781 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:29.779+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:30.135 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:30.133+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:30.740 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:30.738+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:31.179 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:31.177+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:31.757 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:31.755+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:32.210 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:32.208+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:32.724 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:32.722+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:33.208 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:33.206+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:33.714 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:33.712+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:34.187 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:34.185+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:34.746 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:34.744+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:35.176 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:35.174+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:35.774 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:35.772+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:36.192 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:36.190+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:36.782 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:36.780+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:37.230 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:37.228+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:37.785 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:37.783+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:38.197 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:38.195+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:38.782 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:38.780+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:39.233 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:39.231+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:39.785 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:39.783+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:40.215 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:40.213+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:40.744 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:40.742+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:41.228 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:41.225+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:41.790 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:41.788+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:42.187 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:42.185+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:42.758 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:42.755+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:43.155 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:43.153+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:43.778 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:43.776+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:44.198 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:44.196+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:44.747 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:44.745+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:45.239 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:45.236+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:45.710 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:45.708+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:46.258 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:46.256+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:46.709 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:46.707+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:47.262 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:47.260+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:47.747 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:47.745+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:48.272 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:48.270+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:48.740 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:48.738+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:49.300 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:49.297+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:49.697 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:49.695+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:50.300 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:50.299+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:50.651 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:50.649+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:51.299 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:51.297+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:51.625 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:51.623+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:52.320 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:52.318+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:52.645 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:52.643+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:53.367 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:53.366+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:53.668 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:53.666+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:54.381 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:54.380+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:54.633 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:54.632+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:55.427 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:55.426+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:55.645 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:55.644+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:56.395 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:56.394+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:56.603 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:56.602+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:57.426 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:57.425+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:57.579 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:57.578+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:58.473 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:58.472+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:58.579 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:58.578+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:59.498 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:17:59.497+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:17:59.596 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:17:59.595+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:00.517 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:00.516+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:00.593 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:00.592+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:01.481 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:01.480+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:01.549 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:01.548+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:02.444 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:02.443+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:02.508 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:02.506+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:03.477 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:03.476+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:03.503 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:03.502+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:04.439 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:04.438+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:04.522 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:04.521+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:05.408 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:05.406+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:05.483 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:05.482+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:06.375 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:06.374+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:06.477 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:06.476+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:07.327 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:07.326+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:07.469 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:07.468+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:08.294 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:08.293+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:08.488 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:08.487+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:09.265 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:09.264+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:09.443 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:09.442+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:10.235 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:10.234+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:10.458 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:10.457+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:11.211 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:11.210+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:11.459 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:11.458+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:12.187 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:12.186+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:12.439 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:12.438+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:13.147 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:13.146+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:13.453 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:13.451+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:14.157 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:14.155+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:14.439 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:14.437+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:15.118 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:15.116+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:15.490 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:15.487+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:16.101 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:16.099+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:16.490 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:16.488+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:17.121 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:17.119+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:17.511 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:17.509+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:18.114 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:18.112+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:18.551 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:18.549+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:19.141 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:19.138+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:19.515 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:19.513+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:20.159 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:20.157+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:20.533 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:20.531+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:21.183 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:21.181+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:21.551 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:21.549+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:22.137 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:22.134+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:22.505 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:22.503+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:23.177 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:23.175+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:23.486 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:23.484+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:24.209 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:24.207+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:24.443 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:24.441+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:25.221 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:25.219+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:25.488 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:25.486+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:26.254 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:26.252+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:26.440 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:26.438+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:27.250 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:27.248+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:27.411 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:27.409+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:28.232 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:28.230+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:28.382 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:28.380+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:29.251 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:29.249+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:29.378 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:29.376+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:30.223 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:30.221+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:30.357 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:30.354+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:31.203 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:31.201+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:31.328 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:31.325+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:32.189 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:32.187+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:32.307 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:32.305+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:33.197 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:33.195+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:33.293 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:33.291+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:34.155 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:34.153+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:34.299 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:34.297+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:35.188 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:35.185+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:35.271 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:35.269+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:36.238 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:36.236+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:36.269 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:36.267+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:37.238 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:37.236+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:37.258 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:37.256+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:38.225 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:38.223+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:38.291 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:38.289+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:39.247 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:39.245+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:39.341 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:39.339+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:40.212 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:40.210+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:40.305 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:40.303+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:41.237 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:41.235+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:41.318 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:41.316+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:42.190 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:42.188+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:42.316 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:42.314+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:43.203 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:43.201+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:43.269 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:43.267+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:44.190 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:44.188+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:44.259 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:44.257+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:45.156 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:45.154+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:45.210 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:45.208+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:46.149 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:46.147+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:46.168 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:46.166+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:47.167 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:47.165+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:47.178 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:47.176+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:48.119 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:48.117+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:48.177 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:48.175+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:49.138 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:49.136+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:49.153 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:49.151+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:50.148 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:50.146+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:50.153 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:50.151+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:51.108 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:51.106+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:51.123 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:51.121+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:52.133 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:52.132+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:52.137 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:52.136+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:53.110 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:53.109+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:53.173 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:53.172+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:54.071 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:54.070+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:54.139 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:54.138+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:55.119 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:55.118+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:55.149 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:55.148+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:56.152 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:56.151+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:56.163 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:56.162+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:57.153 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:57.152+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:57.235 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:57.187+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:58.127 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:58.126+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:58.202 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:58.201+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:59.137 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:18:59.136+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:18:59.185 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:18:59.184+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:00.159 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:00.158+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:00.205 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:00.204+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:01.185 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:01.184+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:01.222 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:01.221+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:02.152 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:02.150+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:02.262 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:02.261+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:03.181 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:03.179+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:03.291 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:03.290+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:04.206 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:04.205+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:04.257 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:04.256+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:05.175 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:05.174+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:05.261 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:05.260+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:06.136 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:06.135+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:06.231 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:06.230+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:07.114 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:07.113+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:07.219 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:07.218+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:08.070 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:08.068+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:08.263 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:08.262+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:09.053 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:09.052+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:09.245 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:09.244+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:10.039 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:10.038+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:10.291 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:10.290+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:11.003 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:11.002+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:11.320 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:11.319+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:11.958 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:11.956+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:12.304 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:12.302+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:12.968 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:12.966+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:13.345 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:13.343+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:13.986 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:13.984+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:14.370 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:14.368+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:15.027 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:15.025+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:15.623 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:15.361+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:16.011 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:16.009+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:16.339 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:16.337+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:16.967 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:16.965+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:17.351 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:17.349+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:18.014 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:18.011+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:18.617 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:18.316+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:18.974 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:18.972+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:19.294 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:19.292+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:19.991 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:19.989+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:20.254 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:20.252+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:20.959 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:20.956+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:21.303 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:21.301+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:21.995 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:21.992+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:22.314 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:22.311+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:22.995 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:22.993+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:23.296 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:23.294+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:24.042 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:24.040+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:24.281 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:24.279+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:25.072 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:25.070+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:25.304 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:25.302+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:26.031 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:26.028+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:26.352 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:26.350+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:27.013 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:27.011+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:27.397 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:27.394+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:28.057 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:28.055+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:28.439 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:28.437+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:29.053 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:29.051+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:29.405 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:29.403+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:30.061 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:30.059+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:30.445 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:30.443+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:31.015 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:31.013+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:31.468 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:31.466+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:31.981 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:31.979+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:32.514 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:32.512+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:33.011 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:33.009+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:33.545 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:33.543+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:34.011 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:34.009+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:34.513 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:34.511+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:35.051 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:35.049+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:35.562 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:35.560+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:36.099 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:36.097+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:36.583 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:36.581+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:37.139 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:37.137+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:37.571 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:37.569+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:38.175 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:38.173+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:38.605 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:38.603+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:39.192 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:39.189+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:39.608 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:39.606+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:40.219 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:40.217+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:40.566 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:40.564+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:41.199 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:41.197+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:41.609 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:41.607+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:42.186 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:42.184+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:42.648 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:42.646+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:43.139 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:43.137+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:43.636 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:43.634+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:44.167 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:44.165+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:44.604 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:44.602+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:45.184 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:45.182+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:45.629 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:45.627+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:46.171 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:46.169+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:46.672 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:46.670+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:47.142 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:47.140+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:47.676 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:47.674+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:48.151 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:48.149+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:48.691 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:48.689+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:49.156 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:49.154+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:49.676 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:49.674+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:50.167 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:50.166+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:50.703 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:50.702+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:51.211 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:51.210+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:51.698 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:51.697+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:52.162 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:52.161+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:52.742 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:52.741+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:53.206 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:53.205+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:53.764 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:53.763+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:54.249 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:54.245+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:54.786 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:54.785+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:55.234 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:55.232+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:55.740 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:55.739+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:56.198 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:56.197+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:56.740 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:56.739+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:57.220 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:57.219+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:57.743 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:57.742+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:58.179 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:58.177+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:58.768 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:58.767+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:59.175 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:19:59.174+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:19:59.764 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:19:59.763+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:00.127 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:00.126+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:00.778 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:00.777+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:01.166 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:01.165+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:01.748 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:01.747+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:02.122 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:02.121+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:02.772 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:02.771+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:03.082 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:03.081+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:03.736 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:03.735+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:04.072 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:04.071+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:04.742 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:04.741+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:05.025 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:05.023+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:05.766 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:05.764+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:05.997 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:05.996+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:06.793 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:06.791+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:06.995 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:06.994+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:07.802 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:07.801+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:07.998 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:07.997+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:08.772 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:08.771+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:09.025 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:09.024+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:09.792 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:09.790+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:09.982 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:09.981+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:10.816 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:10.814+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:10.935 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:10.933+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:11.832 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:11.830+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:11.934 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:11.932+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:12.837 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:12.834+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:12.914 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:12.912+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:13.860 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:13.858+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:13.947 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:13.945+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:14.874 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:14.872+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:14.930 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:14.928+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:15.876 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:15.874+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:15.889 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:15.887+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:16.828 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:16.826+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:16.906 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:16.904+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:17.867 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:17.865+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:17.933 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:17.931+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:18.863 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:18.861+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:19.191 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:18.909+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:19.838 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:19.836+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:19.892 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:19.890+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:20.859 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:20.857+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:20.932 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:20.930+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:21.909 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:21.907+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:21.982 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:21.980+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:22.903 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:22.901+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:22.935 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:22.934+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:23.896 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:23.893+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:23.939 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:23.936+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:24.890 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:24.888+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:24.972 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:24.970+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:25.910 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:25.908+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:25.952 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:25.950+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:26.941 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:26.939+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:26.971 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:26.969+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:27.906 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:27.904+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:28.006 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:28.004+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:28.954 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:28.952+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:29.057 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:29.055+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:29.915 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:29.912+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:30.102 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:30.099+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:30.943 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:30.941+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:31.106 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:31.103+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:31.948 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:31.946+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:32.061 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:32.059+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:32.988 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:32.986+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:33.092 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:33.090+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:34.025 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:34.023+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:34.114 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:34.112+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:35.065 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:35.063+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:35.124 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:35.122+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:36.018 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:36.016+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:36.140 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:36.138+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:37.035 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:37.033+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:37.148 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:37.146+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:38.022 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:38.020+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:38.185 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:38.183+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:39.020 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:39.018+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:39.232 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:39.230+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:40.029 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:40.027+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:40.196 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:40.194+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:41.054 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:41.052+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:41.164 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:41.162+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:42.084 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:42.082+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:42.192 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:42.190+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:43.079 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:43.077+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:43.225 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:43.223+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:44.068 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:44.066+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:44.254 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:44.252+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:45.065 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:45.063+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:45.224 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:45.222+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:46.074 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:46.072+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:46.211 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:46.209+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:47.093 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:47.091+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:47.169 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:47.168+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:48.056 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:48.054+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:48.164 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:48.161+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:49.105 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:49.103+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:49.125 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:49.124+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:50.070 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:50.069+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:50.116 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:50.116+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:51.081 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:51.080+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:51.104 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:51.103+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:52.055 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:52.054+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:52.080 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:52.079+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:53.022 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:53.021+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:53.068 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:53.067+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:54.024 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:54.023+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:54.026 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:54.025+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:54.982 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:54.981+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:55.052 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:55.051+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:55.939 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:55.938+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:56.091 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:56.090+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:56.950 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:56.949+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:57.071 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:57.070+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:57.959 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:57.958+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:58.051 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:58.050+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:58.912 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:58.911+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:59.043 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:20:59.042+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:20:59.956 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:20:59.955+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:00.085 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:00.084+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:00.939 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:00.938+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:01.101 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:01.100+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:01.922 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:01.921+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:02.134 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:02.133+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:02.929 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:02.928+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:03.131 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:03.130+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:03.944 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:03.943+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:04.163 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:04.161+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:04.931 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:04.930+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:05.140 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:05.139+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:05.955 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:05.954+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:06.189 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:06.188+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:06.947 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:06.946+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:07.217 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:07.216+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:07.957 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:07.956+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:08.260 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:08.259+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:08.915 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:08.913+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:09.295 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:09.293+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:09.949 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:09.947+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:10.345 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:10.343+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:10.901 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:10.899+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:11.353 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:11.351+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:11.931 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:11.929+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:12.364 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:12.362+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:12.899 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:12.897+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:13.398 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:13.396+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:13.930 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:13.927+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:14.447 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:14.445+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:14.909 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:14.906+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:15.409 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:15.407+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:15.956 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:15.954+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:16.377 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:16.375+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:16.962 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:16.960+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:17.386 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:17.384+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:18.000 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:17.998+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:18.347 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:18.345+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:18.992 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:18.990+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:19.315 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:19.313+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:19.978 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:19.945+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:20.304 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:20.301+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:20.939 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:20.937+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:21.326 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:21.324+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:21.936 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:21.934+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:22.286 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:22.284+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:22.949 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:22.947+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:23.699 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:23.246+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:23.989 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:23.987+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:24.285 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:24.283+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:25.038 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:25.036+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:25.240 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:25.238+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:26.051 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:26.049+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:26.233 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:26.230+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:27.044 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:27.042+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:27.266 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:27.264+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:28.024 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:28.021+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:28.234 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:28.232+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:29.026 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:29.024+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:29.271 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:29.269+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:30.060 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:30.058+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:30.239 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:30.237+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:31.059 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:31.057+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:31.231 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:31.229+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:32.077 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:32.074+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:32.273 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:32.271+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:33.059 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:33.057+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:33.251 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:33.249+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:34.109 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:34.107+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:34.229 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:34.227+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:35.129 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:35.127+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:35.261 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:35.259+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:36.128 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:36.126+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:36.280 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:36.278+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:37.175 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:37.173+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:37.255 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:37.253+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:38.152 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:38.150+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:38.217 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:38.215+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:39.107 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:39.105+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:39.236 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:39.234+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:40.085 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:40.082+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:40.189 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:40.187+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:41.119 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:41.117+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:41.186 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:41.183+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:42.108 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:42.106+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:42.137 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:42.135+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:43.091 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:43.089+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:43.133 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:43.131+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:44.090 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:44.088+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:44.147 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:44.145+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:45.089 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:45.087+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:45.180 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:45.178+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:46.138 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:46.136+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:46.203 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:46.201+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:47.122 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:47.121+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:47.170 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:47.169+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:48.129 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:48.128+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:48.159 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:48.158+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:49.156 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:49.155+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:49.208 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:49.207+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:50.116 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:50.115+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:50.244 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:50.243+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:51.118 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:51.117+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:51.231 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:51.230+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:52.156 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:52.155+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:52.253 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:52.251+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:53.176 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:53.175+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:53.215 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:53.214+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:54.139 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:54.138+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:54.188 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:54.187+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:55.143 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:55.142+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:55.203 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:55.202+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:56.151 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:56.150+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:56.208 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:56.207+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:57.122 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:57.121+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:57.254 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:57.253+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:58.170 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:58.169+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:58.288 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:58.287+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:59.192 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:21:59.191+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:21:59.267 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:21:59.266+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:00.157 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:00.156+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:00.292 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:00.291+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:01.114 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:01.113+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:01.285 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:01.284+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:02.109 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:02.108+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:02.258 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:02.257+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:03.088 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:03.087+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:03.217 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:03.216+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:04.094 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:04.093+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:04.232 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:04.231+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:05.059 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:05.058+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:05.216 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:05.215+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:06.018 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:06.017+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:06.210 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:06.209+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:07.048 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:07.047+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:07.182 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:07.181+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:08.015 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:08.013+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:08.232 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:08.229+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:09.023 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:09.021+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:09.213 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:09.211+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:09.992 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:09.990+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:10.216 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:10.214+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:10.985 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:10.982+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:11.241 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:11.239+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:11.960 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:11.958+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:12.286 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:12.284+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:12.931 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:12.929+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:13.318 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:13.316+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:13.890 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:13.888+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:14.346 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:14.344+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:14.917 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:14.915+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:15.342 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:15.340+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:15.885 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:15.883+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:16.316 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:16.314+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:16.892 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:16.890+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:17.302 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:17.300+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:17.889 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:17.887+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:18.288 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:18.286+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:18.908 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:18.906+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:19.309 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:19.307+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:19.942 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:19.940+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:20.335 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:20.333+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:20.956 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:20.954+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:21.356 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:21.354+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:21.946 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:21.944+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:22.406 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:22.404+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:22.938 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:22.936+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:23.390 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:23.388+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:23.962 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:23.960+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:24.346 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:24.343+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:24.962 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:24.958+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:25.358 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:25.355+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:26.780 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:25.926+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:26.781 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:26.318+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:26.945 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:26.943+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:27.296 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:27.293+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:27.954 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:27.952+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:28.294 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:28.292+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:28.947 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:28.945+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:29.293 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:29.291+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:29.939 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:29.936+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:30.296 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:30.294+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:30.952 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:30.950+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:31.262 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:31.260+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:31.904 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:31.901+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:32.274 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:32.272+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:32.949 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:32.947+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:33.300 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:33.298+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:33.998 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:33.996+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:34.255 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:34.252+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:34.962 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:34.960+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:35.226 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:35.224+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:35.972 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:35.970+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:36.269 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:36.267+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:37.022 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:37.020+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:37.306 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:37.304+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:37.986 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:37.984+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:38.294 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:38.292+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:39.018 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:39.016+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:39.282 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:39.280+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:39.992 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:39.990+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:40.266 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:40.264+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:41.013 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:41.010+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:41.304 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:41.302+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:41.988 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:41.986+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:42.315 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:42.313+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:42.964 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:42.962+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:43.300 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:43.298+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:43.945 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:43.944+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:44.272 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:44.270+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:44.966 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:44.963+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:45.290 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:45.289+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:45.948 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:45.947+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:46.308 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:46.307+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:46.965 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:46.963+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:47.312 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:47.311+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:48.015 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:48.014+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:48.325 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:48.323+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:48.991 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:48.990+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:49.366 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:49.365+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:49.976 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:49.975+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:50.320 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:50.319+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:50.972 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:50.971+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:51.275 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:51.274+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:51.975 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:51.974+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:52.293 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:52.292+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:52.930 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:52.929+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:53.325 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:53.324+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:53.897 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:53.895+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:54.284 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:54.283+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:54.895 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:54.894+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:55.293 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:55.292+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:55.897 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:55.896+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:56.328 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:56.327+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:56.943 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:56.942+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:57.333 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:57.332+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:57.909 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:57.908+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:58.316 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:58.315+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:58.936 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:58.935+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:59.343 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:22:59.342+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:22:59.955 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:22:59.954+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:00.298 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:00.297+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:00.925 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:00.923+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:01.303 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:01.302+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:01.959 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:01.958+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:02.280 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:02.279+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:02.925 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:02.924+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:03.302 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:03.301+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:03.954 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:03.952+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:04.337 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:04.335+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:04.969 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:04.968+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:05.307 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:05.306+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:05.949 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:05.946+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:06.312 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:06.310+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:06.981 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:06.979+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:07.349 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:07.346+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:07.977 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:07.975+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:08.369 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:08.367+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:08.997 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:08.994+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:09.379 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:09.377+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:09.987 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:09.985+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:10.397 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:10.395+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:10.975 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:10.973+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:11.358 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:11.356+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:11.997 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:11.995+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:12.389 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:12.387+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:12.981 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:12.979+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:13.399 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:13.397+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:13.995 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:13.993+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:14.427 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:14.425+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:15.020 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:15.018+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:15.457 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:15.455+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:16.029 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:16.027+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:16.413 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:16.411+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:17.039 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:17.037+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:17.416 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:17.414+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:18.015 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:18.013+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:18.372 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:18.370+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:19.061 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:19.059+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:19.389 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:19.386+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:20.043 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:20.041+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:20.344 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:20.342+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:21.049 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:21.046+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:21.369 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:21.367+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:22.017 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:22.015+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:22.395 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:22.393+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:22.977 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:22.975+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:23.358 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:23.356+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:23.929 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:23.927+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:24.333 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:24.331+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:24.927 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:24.925+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:25.349 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:25.347+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:25.978 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:25.975+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:26.390 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:26.388+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:26.939 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:26.937+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:27.366 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:27.364+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:27.921 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:27.919+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:28.349 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:28.347+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:28.874 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:28.872+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:29.325 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:29.323+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:29.876 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:29.874+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:30.367 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:30.365+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:30.924 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:30.922+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:31.321 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:31.319+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:31.931 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:31.929+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:32.287 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:32.285+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:32.980 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:32.978+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:33.334 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:33.332+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:34.002 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:33.999+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:34.379 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:34.377+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:34.985 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:34.983+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:35.407 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:35.405+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:36.034 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:36.031+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:36.391 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:36.389+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:37.071 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:37.069+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:37.372 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:37.370+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:38.109 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:38.107+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:38.341 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:38.339+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:39.146 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:39.144+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:39.364 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:39.362+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:40.109 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:40.107+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:40.385 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:40.383+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:41.072 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:41.069+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:41.389 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:41.387+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:42.071 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:42.069+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:42.393 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:42.391+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:43.103 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:43.101+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:43.350 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:43.348+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:44.109 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:44.108+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:44.364 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:44.363+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:45.068 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:45.067+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:45.385 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:45.384+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:46.092 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:46.091+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:46.429 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:46.428+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:47.055 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:47.054+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:47.383 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:47.382+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:48.082 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:48.081+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:48.416 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:48.415+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:49.054 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:49.053+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:49.376 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:49.374+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:50.044 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:50.043+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:50.359 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:50.358+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:51.070 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:51.069+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:51.328 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:51.327+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:52.106 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:52.105+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:52.323 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:52.322+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:53.149 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:53.148+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:53.340 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:53.339+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:54.125 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:54.124+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:54.354 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:54.353+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:55.160 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:55.159+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:55.320 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:55.319+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:56.205 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:56.204+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:56.290 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:56.289+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:57.226 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:57.225+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:57.290 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:57.289+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:58.205 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:58.204+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:58.315 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:58.314+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:59.189 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:23:59.188+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:23:59.302 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:23:59.301+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:00.216 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:00.215+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:00.343 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:00.342+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:01.193 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:01.192+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:01.350 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:01.349+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:02.188 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:02.187+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:02.349 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:02.348+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:03.209 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:03.208+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:03.321 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:03.320+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:04.248 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:04.245+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:04.274 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:04.272+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:05.216 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:05.214+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:05.226 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:05.224+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:06.224 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:06.222+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:06.225 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:06.223+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:07.207 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:07.205+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:07.272 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:07.270+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:08.246 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:08.244+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:08.286 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:08.284+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:09.241 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:09.239+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:09.285 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:09.283+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:10.197 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:10.195+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:10.322 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:10.319+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:11.195 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:11.193+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:11.294 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:11.292+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:12.244 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:12.242+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:12.250 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:12.249+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:13.241 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:13.238+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:13.288 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:13.286+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:14.262 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:14.260+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:14.272 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:14.270+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:15.290 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:15.288+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:15.307 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:15.305+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:16.264 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:16.262+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:16.286 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:16.284+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:17.296 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:17.294+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:17.298 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:17.296+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:18.282 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:18.280+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:18.296 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:18.294+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:19.259 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:19.257+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:19.290 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:19.288+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:20.230 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:20.228+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:20.284 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:20.281+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:21.224 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:21.222+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:21.238 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:21.236+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:22.265 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:22.263+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:22.269 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:22.268+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:23.291 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:23.289+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:23.296 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:23.294+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:24.283 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:24.281+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:24.296 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:24.294+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:25.266 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:25.263+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:25.309 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:25.307+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:26.276 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:26.274+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:26.306 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:26.304+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:27.262 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:27.260+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:27.278 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:27.276+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:28.267 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:28.265+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:28.303 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:28.301+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:29.261 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:29.259+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:29.262 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:29.260+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:30.244 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:30.242+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:30.304 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:30.302+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:31.258 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:31.256+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:31.340 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:31.338+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:32.263 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:32.261+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:32.327 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:32.325+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:33.256 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:33.254+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:33.377 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:33.375+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:34.234 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:34.231+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:34.422 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:34.420+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:35.244 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:35.242+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:35.378 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:35.376+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:36.236 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:36.234+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:36.424 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:36.421+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:37.189 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:37.186+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:37.380 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:37.378+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:38.228 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:38.226+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:38.376 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:38.374+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:39.193 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:39.192+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:39.390 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:39.388+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:40.164 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:40.162+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:40.359 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:40.358+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:41.196 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:41.193+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:41.390 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:41.388+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:42.153 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:42.152+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:42.377 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:42.376+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:43.188 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:43.187+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:43.400 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:43.399+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:44.166 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:44.165+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:44.389 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:44.388+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:45.124 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:45.123+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:45.358 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:45.356+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:46.146 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:46.145+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:46.357 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:46.356+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:47.140 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:47.139+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:47.331 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:47.330+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:48.100 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:48.099+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:48.287 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:48.285+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:49.150 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:49.149+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:49.319 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:49.318+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:50.143 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:50.141+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:50.324 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:50.323+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:51.165 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:51.164+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:51.353 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:51.353+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:52.214 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:52.213+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:52.338 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:52.336+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:53.247 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:53.246+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:53.374 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:53.373+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:54.243 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:54.242+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:54.372 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:54.370+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:55.279 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:55.278+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:55.419 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:55.418+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:56.239 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:56.238+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:56.453 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:56.452+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:57.198 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:57.197+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:57.427 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:57.426+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:58.191 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:58.190+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:58.430 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:58.429+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:59.221 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:24:59.220+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:24:59.422 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:24:59.421+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:00.217 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:00.216+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:00.376 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:00.375+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:01.216 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:01.215+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:01.346 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:01.345+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:02.177 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:02.176+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:02.318 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:02.316+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:03.167 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:03.165+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:03.333 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:03.331+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:04.208 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:04.206+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:04.336 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:04.334+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:05.232 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:05.230+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:05.376 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:05.374+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:06.191 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:06.189+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:06.379 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:06.377+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:07.176 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:07.173+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:07.345 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:07.343+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:08.216 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:08.214+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:08.392 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:08.390+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:09.210 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:09.208+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:09.434 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:09.431+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:10.249 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:10.247+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:10.454 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:10.452+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:11.238 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:11.235+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:11.459 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:11.457+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:12.220 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:12.218+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:12.424 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:12.422+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:13.184 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:13.182+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:13.390 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:13.388+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:14.212 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:14.210+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:14.440 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:14.438+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:15.190 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:15.188+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:15.466 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:15.464+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:16.237 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:16.235+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:16.466 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:16.464+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:17.254 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:17.252+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:17.464 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:17.462+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:18.209 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:18.207+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:18.430 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:18.428+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:19.221 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:19.219+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:19.429 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:19.427+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:20.234 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:20.232+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:20.416 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:20.414+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:21.261 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:21.259+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:21.394 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:21.392+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:22.288 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:22.286+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:22.361 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:22.360+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:23.275 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:23.272+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:23.320 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:23.318+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:24.274 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:24.272+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:24.318 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:24.316+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:25.309 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:25.307+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:25.332 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:25.330+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:26.303 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:26.301+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:26.334 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:26.332+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:27.328 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:27.326+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:27.350 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:27.348+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:28.312 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:28.310+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:28.345 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:28.344+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:29.310 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:29.308+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:29.319 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:29.318+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:30.272 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:30.270+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:30.318 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:30.316+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:31.269 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:31.266+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:31.294 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:31.291+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:32.310 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:32.308+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:32.327 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:32.326+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:33.270 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:33.268+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:33.370 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:33.368+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:34.296 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:34.294+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:34.332 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:34.330+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:35.252 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:35.250+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:35.371 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:35.369+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:36.243 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:36.240+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:36.352 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:36.350+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:37.250 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:37.247+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:37.340 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:37.338+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:38.279 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:38.277+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:38.348 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:38.346+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:39.310 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:39.308+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:39.322 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:39.320+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:40.278 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:40.277+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:40.346 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:40.345+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:41.285 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:41.283+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:41.328 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:41.327+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:42.281 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:42.280+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:42.318 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:42.317+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:43.303 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:43.302+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:43.352 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:43.351+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:44.312 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:44.311+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:44.321 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:44.320+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:45.314 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:45.313+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:45.368 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:45.367+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:46.283 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:46.282+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:46.372 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:46.371+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:47.258 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:47.257+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:47.368 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:47.367+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:48.217 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:48.216+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:48.361 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:48.360+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:49.178 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:49.177+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:49.401 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:49.400+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:50.205 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:50.204+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:50.438 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:50.437+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:51.202 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:51.201+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:51.448 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:51.447+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:52.162 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:52.161+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:52.448 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:52.447+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:53.202 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:53.201+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:53.448 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:53.447+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:54.233 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:54.232+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:54.450 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:54.449+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:55.282 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:55.281+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:55.422 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:55.421+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:56.307 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:56.306+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:56.458 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:56.457+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:57.356 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:57.355+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:57.442 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:57.441+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:58.350 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:58.349+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:58.472 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:58.471+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:59.309 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:25:59.308+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:25:59.434 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:25:59.433+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:00.300 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:00.298+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:00.437 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:00.436+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:01.266 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:01.264+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:01.480 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:01.478+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:02.311 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:02.309+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:02.445 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:02.443+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:03.334 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:03.332+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:03.489 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:03.487+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:04.289 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:04.287+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:04.450 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:04.448+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:05.304 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:05.302+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:05.439 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:05.437+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:06.296 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:06.294+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:06.487 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:06.485+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:07.266 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:07.264+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:07.530 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:07.527+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:08.317 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:08.315+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:08.533 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:08.531+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:09.334 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:09.332+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:09.492 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:09.490+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:10.346 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:10.344+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:10.448 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:10.446+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:11.338 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:11.335+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:11.434 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:11.431+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:12.349 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:12.347+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:12.409 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:12.407+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:13.385 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:13.383+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:13.444 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:13.442+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:14.342 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:14.340+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:14.399 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:14.397+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:15.318 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:15.316+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:15.361 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:15.359+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:16.307 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:16.305+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:16.313 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:16.312+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:17.273 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:17.271+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:17.342 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:17.340+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:18.304 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:18.302+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:18.366 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:18.364+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:19.332 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:19.330+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:19.365 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:19.363+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:20.381 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:20.379+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:20.390 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:20.388+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:21.353 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:21.351+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:21.363 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:21.361+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:22.308 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:22.306+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:22.368 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:22.366+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:23.329 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:23.327+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:23.343 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:23.341+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:24.328 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:24.325+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:24.368 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:24.365+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:25.346 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:25.344+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:25.390 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:25.388+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:26.316 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:26.314+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:26.441 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:26.439+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:27.294 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:27.292+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:27.426 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:27.424+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:28.343 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:28.341+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:28.449 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:28.447+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:29.297 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:29.295+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:29.466 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:29.464+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:30.342 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:30.340+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:30.510 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:30.508+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:31.314 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:31.312+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:31.496 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:31.494+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:32.329 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:32.327+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:32.537 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:32.535+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:33.303 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:33.301+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:33.557 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:33.555+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:34.307 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:34.305+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:34.514 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:34.512+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:35.269 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:35.267+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:35.469 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:35.467+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:36.311 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:36.309+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:36.433 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:36.431+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:37.358 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:37.356+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:37.440 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:37.439+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:38.400 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:38.399+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:38.489 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:38.487+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:39.379 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:39.378+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:39.529 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:39.528+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:40.365 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:40.364+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:40.754 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:40.515+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:41.410 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:41.409+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:41.528 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:41.527+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:42.459 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:42.457+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:42.661 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:42.576+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:43.425 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:43.424+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:43.618 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:43.617+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:44.449 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:44.448+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:44.571 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:44.570+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:45.439 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:45.438+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:45.541 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:45.540+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:46.402 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:46.401+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:46.538 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:46.537+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:47.421 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:47.419+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:47.588 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:47.587+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:48.471 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:48.469+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:48.599 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:48.598+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:49.443 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:49.442+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:49.570 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:49.568+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:50.420 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:50.419+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:50.558 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:50.557+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:51.394 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:51.392+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:51.545 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:51.544+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:52.416 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:52.415+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:52.542 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:52.541+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:53.385 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:53.384+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:53.561 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:53.560+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:54.431 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:54.430+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:54.605 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:54.604+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:55.389 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:55.388+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:55.560 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:55.559+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:56.367 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:56.366+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:56.571 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:56.570+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:57.345 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:57.344+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:57.608 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:57.607+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:58.335 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:58.333+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:58.618 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:58.617+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:59.341 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:26:59.339+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:26:59.583 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:26:59.581+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:00.321 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:00.319+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:00.549 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:00.547+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:01.329 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:01.327+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:01.511 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:01.509+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:02.325 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:02.323+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:02.489 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:02.487+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:03.281 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:03.278+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:03.476 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:03.474+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:04.276 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:04.274+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:04.457 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:04.455+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:05.267 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:05.265+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:05.427 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:05.425+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:06.313 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:06.310+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:06.469 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:06.467+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:07.315 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:07.313+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:07.443 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:07.441+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:08.345 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:08.343+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:08.457 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:08.455+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:09.306 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:09.304+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:09.497 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:09.494+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:10.286 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:10.284+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:10.485 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:10.483+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:11.294 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:11.292+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:11.465 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:11.463+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:12.317 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:12.315+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:12.427 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:12.425+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:13.351 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:13.349+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:13.413 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:13.411+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:14.348 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:14.346+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:14.433 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:14.431+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:15.371 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:15.369+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:15.417 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:15.415+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:16.367 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:16.365+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:16.411 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:16.409+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:17.374 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:17.372+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:17.394 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:17.393+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:18.385 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:18.383+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:18.444 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:18.442+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:19.389 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:19.387+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:19.468 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:19.466+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:20.348 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:20.346+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:20.435 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:20.433+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:21.332 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:21.330+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:21.410 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:21.407+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:22.314 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:22.312+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:22.397 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:22.395+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:23.363 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:23.361+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:23.365 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:23.363+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:24.370 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:24.368+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:24.382 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:24.380+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:25.347 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:25.345+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:25.365 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:25.363+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:26.327 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:26.325+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:26.389 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:26.386+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:27.318 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:27.315+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:27.407 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:27.405+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:28.336 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:28.333+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:28.443 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:28.441+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:29.341 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:29.339+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:29.449 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:29.447+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:30.355 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:30.353+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:30.405 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:30.403+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:31.317 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:31.315+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:31.389 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:31.386+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:32.313 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:32.311+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:32.403 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:32.401+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:33.306 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:33.303+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:33.410 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:33.408+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:34.271 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:34.269+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:34.458 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:34.456+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:35.240 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:35.239+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:35.492 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:35.490+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:36.254 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:36.252+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:36.455 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:36.453+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:37.292 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:37.291+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:37.425 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:37.424+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:38.283 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:38.282+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:38.389 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:38.388+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:39.257 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:39.256+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:39.369 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:39.368+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:40.299 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:40.298+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:40.331 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:40.330+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:41.273 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:41.272+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:41.375 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:41.374+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:42.285 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:42.284+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:42.423 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:42.422+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:43.278 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:43.277+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:43.375 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:43.374+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:44.307 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:44.305+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:44.344 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:44.343+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:45.327 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:45.326+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:45.354 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:45.353+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:46.343 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:46.342+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:46.347 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:46.346+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:47.319 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:47.318+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:47.336 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:47.335+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:48.360 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:48.359+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:48.403 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:48.384+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:49.341 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:49.339+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:49.378 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:49.377+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:50.309 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:50.308+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:50.348 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:50.347+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:51.273 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:51.272+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:51.374 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:51.373+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:52.247 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:52.246+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:52.387 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:52.386+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:53.198 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:53.197+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:53.355 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:53.354+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:54.235 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:54.233+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:54.377 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:54.375+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:55.230 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:55.229+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:55.365 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:55.364+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:56.276 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:56.275+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:56.386 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:56.385+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:57.255 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:57.253+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:57.349 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:57.347+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:58.266 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:58.263+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:58.309 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:58.307+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:59.298 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:27:59.296+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:27:59.317 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:27:59.315+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:00.305 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:00.303+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:00.320 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:00.318+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:01.290 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:01.288+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:01.354 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:01.352+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:02.327 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:02.325+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:02.352 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:02.350+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:03.309 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:03.307+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:03.309 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:03.308+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:04.261 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:04.259+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:04.302 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:04.300+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:05.225 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:05.223+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:05.331 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:05.329+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:06.189 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:06.187+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:06.342 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:06.340+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:07.176 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:07.174+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:07.316 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:07.314+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:08.134 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:08.132+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:08.305 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:08.301+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:09.159 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:09.157+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:09.353 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:09.351+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:10.154 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:10.123+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:10.345 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:10.343+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:11.170 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:11.165+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:11.304 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:11.302+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:12.214 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:12.212+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:12.353 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:12.351+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:13.242 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:13.240+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:13.377 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:13.375+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:14.265 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:14.262+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:14.379 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:14.377+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:15.302 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:15.300+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:15.403 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:15.401+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:16.336 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:16.334+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:16.408 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:16.406+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:17.351 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:17.349+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:17.391 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:17.389+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:18.346 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:18.344+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:18.382 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:18.380+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:19.356 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:19.354+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:19.433 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:19.431+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:20.354 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:20.352+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:20.417 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:20.415+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:21.382 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:21.380+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:21.396 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:21.394+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:22.394 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:22.391+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:22.438 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:22.435+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:23.359 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:23.357+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:23.481 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:23.479+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:24.409 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:24.407+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:24.497 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:24.495+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:25.367 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:25.365+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:25.468 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:25.466+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:26.399 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:26.397+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:26.472 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:26.470+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:27.433 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:27.431+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:27.475 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:27.473+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:28.456 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:28.453+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:28.460 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:28.458+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:29.416 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:29.414+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:29.429 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:29.427+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:30.456 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:30.454+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:30.457 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:30.455+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:31.463 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:31.461+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:31.487 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:31.485+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:32.455 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:32.453+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:32.512 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:32.510+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:33.451 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:33.449+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:33.494 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:33.492+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:34.463 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:34.460+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:34.500 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:34.499+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:35.469 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:35.468+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:35.499 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:35.498+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:36.514 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:36.513+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:36.520 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:36.519+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:37.494 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:37.493+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:37.506 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:37.505+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:38.524 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:38.523+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:38.555 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:38.554+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:39.512 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:39.511+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:39.524 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:39.523+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:40.467 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:40.466+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:40.536 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:40.535+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:41.462 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:41.461+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:41.547 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:41.546+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:42.465 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:42.464+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:42.592 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:42.591+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:43.497 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:43.496+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:43.633 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:43.632+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:44.528 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:44.527+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:44.663 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:44.662+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:45.504 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:45.503+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:45.639 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:45.638+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:46.456 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:46.455+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:46.681 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:46.680+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:47.446 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:47.445+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:47.658 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:47.657+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:48.412 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:48.411+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:48.706 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:48.705+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:49.429 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:49.428+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:49.658 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:49.657+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:50.434 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:50.433+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:50.686 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:50.685+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:51.408 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:51.407+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:51.669 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:51.668+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:52.423 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:52.422+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:52.667 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:52.666+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:53.460 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:53.459+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:53.651 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:53.650+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:54.476 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:54.475+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:54.647 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:54.646+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:55.482 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:55.480+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:55.606 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:55.604+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:56.438 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:56.436+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:56.602 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:56.600+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:57.451 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:57.449+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:57.606 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:57.604+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:58.425 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:58.423+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:58.584 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:58.582+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:59.448 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:28:59.446+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:28:59.578 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:28:59.576+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:00.460 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:00.458+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:00.597 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:00.595+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:01.452 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:01.450+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:01.594 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:01.592+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:02.500 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:02.498+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:02.589 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:02.587+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:03.524 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:03.522+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:03.630 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:03.628+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:04.518 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:04.516+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:04.666 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:04.664+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:05.550 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:05.547+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:05.625 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:05.623+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:06.570 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:06.567+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:06.588 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:06.586+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:07.542 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:07.540+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:07.610 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:07.608+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:08.498 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:08.496+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:08.643 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:08.641+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:09.452 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:09.450+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:09.678 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:09.676+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:10.425 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:10.423+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:10.692 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:10.690+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:11.412 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:11.410+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:11.672 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:11.670+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:12.365 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:12.362+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:12.629 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:12.624+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:13.402 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:13.400+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:13.664 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:13.662+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:14.376 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:14.374+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:14.643 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:14.641+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:15.392 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:15.390+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:15.689 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:15.686+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:16.360 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:16.358+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:16.663 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:16.661+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:17.316 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:17.314+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:17.646 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:17.644+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:18.273 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:18.271+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:18.685 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:18.683+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:19.313 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:19.311+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:19.654 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:19.652+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:20.270 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:20.268+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:20.608 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:20.606+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:21.226 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:21.224+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:21.658 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:21.656+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:22.209 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:22.207+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:22.614 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:22.612+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:23.173 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:23.171+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:23.610 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:23.608+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:24.170 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:24.168+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:24.612 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:24.610+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:25.204 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:25.202+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:25.609 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:25.607+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:26.224 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:26.222+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:26.581 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:26.579+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:27.213 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:27.211+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:27.533 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:27.531+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:28.262 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:28.260+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:28.491 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:28.489+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:29.217 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:29.214+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:29.448 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:29.446+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:30.188 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:30.186+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:30.401 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:30.399+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:31.152 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:31.150+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:31.441 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:31.439+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:32.172 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:32.170+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:32.450 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:32.448+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:33.159 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:33.158+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:33.443 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:33.442+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:34.201 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:34.199+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:34.422 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:34.421+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:35.216 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:35.215+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:35.456 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:35.455+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:36.219 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:36.218+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:36.472 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:36.471+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:37.196 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:37.195+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:37.516 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:37.515+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:38.235 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:38.234+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:38.492 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:38.491+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:39.264 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:39.263+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:39.475 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:39.474+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:40.293 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:40.292+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:40.467 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:40.466+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:41.344 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:41.343+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:41.463 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:41.462+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:42.335 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:42.334+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:42.494 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:42.493+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:43.321 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:43.320+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:43.503 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:43.502+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:44.319 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:44.318+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:44.543 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:44.542+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:45.310 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:45.309+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:45.589 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:45.588+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:46.292 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:46.291+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:46.604 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:46.603+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:47.336 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:47.335+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:47.610 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:47.609+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:48.341 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:48.340+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:48.626 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:48.625+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:49.336 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:49.335+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:49.596 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:49.595+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:50.360 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:50.359+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:50.564 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:50.563+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:51.333 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:51.332+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:51.517 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:51.516+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:52.287 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:52.286+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:52.549 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:52.548+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:53.266 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:53.264+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:53.570 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:53.568+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:54.221 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:54.219+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:54.573 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:54.571+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:55.192 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:55.190+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:55.530 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:55.528+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:56.214 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:56.212+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:56.519 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:56.517+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:57.197 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:57.195+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:57.558 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:57.556+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:58.172 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:58.170+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:58.576 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:58.574+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:59.218 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:29:59.216+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:29:59.570 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:29:59.568+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:00.186 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:00.184+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:00.608 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:00.606+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:01.233 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:01.231+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:01.650 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:01.648+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:02.281 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:02.279+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:02.620 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:02.617+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:03.265 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:03.263+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:03.665 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:03.662+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:04.231 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:04.229+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:04.690 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:04.688+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:05.196 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:05.194+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:05.692 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:05.690+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:06.224 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:06.222+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:06.696 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:06.694+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:07.212 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:07.210+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:07.684 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:07.681+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:08.164 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:08.162+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:08.714 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:08.712+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:09.118 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:09.116+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:09.724 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:09.722+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:10.070 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:10.067+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:10.738 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:10.736+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:11.026 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:11.024+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:11.788 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:11.786+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:12.048 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:12.046+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:12.751 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:12.749+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:13.041 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:13.039+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:13.764 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:13.762+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:14.081 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:14.079+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:14.794 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:14.792+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:15.128 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:15.126+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:15.802 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:15.800+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:16.111 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:16.109+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:16.841 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:16.839+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:17.134 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:17.132+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:17.852 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:17.850+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:18.106 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:18.104+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:18.894 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:18.892+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:19.140 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:19.138+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:19.920 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:19.918+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:20.146 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:20.144+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:20.888 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:20.886+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:21.182 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:21.180+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:21.906 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:21.904+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:22.152 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:22.150+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:22.920 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:22.918+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:23.156 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:23.154+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:23.962 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:23.960+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:24.138 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:24.136+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:24.927 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:24.924+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:25.093 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:25.091+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:25.896 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:25.894+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:26.102 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:26.100+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:26.926 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:26.923+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:27.075 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:27.073+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:27.953 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:27.951+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:28.104 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:28.102+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:28.923 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:28.921+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:29.061 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:29.059+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:29.910 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:29.908+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:30.099 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:30.097+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:30.903 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:30.901+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:31.058 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:31.057+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:31.945 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:31.944+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:32.056 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:32.054+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:32.994 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:32.993+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:33.008 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:33.007+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:33.959 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:33.958+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:34.039 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:34.038+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:34.955 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:34.954+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:35.041 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:35.040+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:35.984 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:35.983+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:36.013 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:36.012+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:37.011 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:37.010+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:37.023 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:37.022+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:37.995 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:37.994+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:38.061 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:38.060+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:38.985 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:38.984+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:39.020 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:39.019+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:39.989 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:39.988+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:40.043 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:40.042+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:40.958 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:40.957+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:41.052 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:41.051+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:41.977 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:41.976+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:42.075 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:42.074+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:43.021 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:43.019+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:43.111 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:43.110+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:44.013 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:44.012+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:44.108 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:44.107+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:45.023 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:45.022+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:45.158 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:45.157+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:46.005 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:46.004+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:46.192 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:46.191+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:46.958 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:46.956+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:47.221 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:47.220+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:47.982 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:47.981+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:48.249 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:48.248+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:48.934 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:48.933+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:49.242 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:49.241+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:49.974 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:49.973+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:50.254 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:50.252+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:50.946 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:50.944+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:51.257 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:51.256+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:51.962 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:51.960+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:52.234 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:52.232+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:52.927 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:52.925+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:53.186 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:53.184+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:53.923 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:53.921+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:54.223 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:54.221+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:54.909 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:54.907+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:55.217 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:55.214+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:55.897 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:55.895+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:56.195 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:56.193+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:56.864 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:56.862+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:57.201 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:57.199+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:57.851 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:57.849+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:58.235 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:58.233+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:58.863 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:58.861+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:59.202 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:30:59.200+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:30:59.877 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:30:59.875+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:00.239 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:00.237+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:00.894 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:00.892+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:01.257 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:01.254+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:01.869 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:01.867+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:02.283 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:02.281+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:02.898 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:02.896+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:03.265 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:03.263+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:03.883 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:03.881+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:04.315 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:04.313+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:04.844 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:04.842+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:05.310 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:05.308+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:05.853 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:05.851+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:06.344 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:06.342+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:06.813 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:06.811+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:07.311 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:07.309+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:07.767 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:07.765+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:08.342 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:08.340+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:08.797 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:08.795+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:09.387 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:09.385+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:09.752 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:09.750+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:10.345 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:10.343+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:10.731 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:10.729+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:11.390 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:11.388+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:11.692 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:11.690+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:12.348 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:12.346+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:12.724 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:12.722+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:13.344 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:13.342+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:13.692 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:13.690+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:14.334 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:14.332+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:14.670 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:14.668+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:15.339 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:15.337+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:15.698 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:15.696+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:16.345 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:16.343+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:16.703 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:16.701+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:17.304 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:17.302+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:17.708 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:17.706+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:18.322 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:18.320+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:18.689 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:18.687+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:19.343 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:19.341+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:19.738 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:19.736+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:20.362 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:20.360+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:20.731 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:20.729+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:21.333 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:21.331+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:21.690 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:21.688+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:22.341 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:22.339+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:22.680 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:22.678+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:23.362 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:23.360+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:23.720 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:23.718+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:24.369 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:24.367+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:24.681 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:24.679+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:25.328 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:25.326+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:25.658 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:25.656+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:26.351 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:26.349+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:26.612 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:26.609+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:27.377 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:27.375+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:27.617 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:27.616+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:28.349 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:28.347+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:28.647 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:28.645+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:29.374 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:29.373+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:29.629 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:29.627+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:30.383 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:30.381+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:30.591 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:30.590+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:31.349 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:31.348+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:31.586 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:31.585+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:32.380 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:32.379+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:32.558 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:32.557+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:33.429 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:33.428+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:33.608 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:33.607+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:34.403 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:34.402+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:34.583 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:34.582+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:35.409 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:35.408+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:35.596 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:35.595+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:36.425 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:36.423+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:36.582 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:36.581+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:37.394 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:37.393+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:37.556 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:37.555+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:38.428 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:38.427+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:38.594 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:38.593+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:39.417 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:39.416+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:39.546 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:39.545+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:40.372 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:40.371+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:40.538 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:40.536+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:41.382 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:41.380+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:41.492 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:41.491+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:42.355 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:42.354+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:42.460 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:42.459+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:43.331 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:43.330+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:43.476 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:43.475+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:44.325 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:44.324+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:44.462 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:44.461+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:45.316 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:45.315+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:45.420 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:45.419+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:46.277 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:46.276+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:46.439 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:46.438+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:47.242 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:47.241+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:47.482 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:47.481+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:48.228 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:48.226+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:48.532 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:48.531+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:49.195 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:49.194+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:49.571 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:49.569+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:50.153 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:50.151+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:50.602 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:50.600+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:51.111 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:51.109+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:51.614 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:51.612+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:52.090 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:52.088+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:52.566 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:52.564+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:53.088 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:53.086+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:53.523 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:53.521+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:54.094 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:54.092+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:54.570 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:54.567+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:55.062 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:55.057+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:55.545 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:55.543+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:56.086 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:56.084+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:56.510 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:56.508+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:57.084 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:57.082+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:57.540 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:57.538+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:58.088 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:58.085+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:58.521 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:58.519+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:59.056 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:31:59.054+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:31:59.524 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:31:59.521+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:00.008 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:00.006+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:00.502 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:00.500+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:00.960 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:00.958+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:01.538 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:01.536+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:01.952 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:01.950+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:02.545 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:02.543+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:02.954 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:02.952+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:03.532 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:03.530+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:03.955 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:03.953+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:04.513 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:04.511+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:04.998 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:04.996+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:05.560 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:05.558+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:06.024 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:06.022+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:06.555 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:06.553+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:07.042 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:07.040+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:07.596 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:07.594+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:08.064 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:08.062+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:08.554 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:08.552+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:09.028 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:09.026+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:09.553 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:09.551+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:10.044 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:10.042+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:10.569 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:10.567+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:11.006 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:11.004+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:11.520 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:11.518+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:11.970 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:11.967+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:12.546 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:12.544+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:12.938 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:12.936+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:13.562 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:13.560+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:13.979 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:13.977+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:14.546 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:14.544+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:14.959 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:14.957+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:15.534 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:15.532+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:15.914 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:15.912+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:16.512 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:16.510+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:16.946 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:16.943+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:17.486 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:17.484+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:17.907 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:17.906+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:18.504 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:18.502+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:18.922 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:18.920+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:19.462 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:19.460+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:19.936 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:19.934+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:20.424 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:20.422+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:20.946 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:20.944+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:21.467 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:21.465+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:21.964 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:21.962+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:22.514 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:22.512+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:22.966 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:22.964+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:23.542 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:23.540+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:24.006 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:24.004+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:24.584 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:24.582+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:25.015 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:25.013+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:25.547 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:25.545+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:25.995 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:25.993+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:26.564 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:26.562+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:27.017 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:27.015+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:27.586 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:27.585+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:28.068 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:28.067+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:28.622 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:28.621+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:29.032 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:29.031+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:29.659 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:29.658+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:30.078 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:30.077+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:30.659 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:30.658+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:31.068 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:31.067+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:31.659 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:31.658+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:32.052 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:32.050+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:32.692 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:32.691+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:33.030 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:33.029+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:33.676 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:33.675+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:34.056 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:34.054+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:34.658 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:34.656+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:35.075 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:35.074+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:35.689 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:35.688+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:36.119 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:36.118+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:36.677 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:36.676+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:37.110 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:37.109+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:37.671 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:37.670+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:38.133 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:38.132+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:38.676 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:38.675+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:39.102 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:39.101+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:39.662 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:39.660+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:40.061 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:40.060+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:40.634 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:40.633+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:41.033 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:41.032+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:41.592 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:41.591+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:42.042 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:42.040+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:42.558 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:42.557+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:43.024 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:43.023+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:43.532 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:43.531+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:44.073 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:44.072+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:44.486 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:44.485+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:45.094 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:45.093+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:45.518 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:45.517+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:46.114 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:46.113+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:46.532 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:46.531+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:47.106 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:47.105+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:47.528 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:47.527+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:48.106 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:48.104+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:48.503 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:48.501+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:49.133 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:49.131+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:49.506 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:49.504+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:50.110 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:50.108+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:50.517 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:50.515+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:51.158 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:51.155+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:51.560 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:51.558+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:52.117 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:52.115+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:52.576 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:52.573+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:53.136 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:53.134+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:53.560 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:53.558+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:54.180 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:54.178+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:54.580 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:54.578+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:55.168 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:55.166+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:55.566 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:55.564+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:56.190 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:56.188+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:56.548 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:56.546+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:57.194 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:57.192+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:57.530 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:57.528+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:58.152 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:58.150+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:58.580 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:58.578+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:59.116 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:32:59.114+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:32:59.589 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:32:59.587+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:00.134 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:00.132+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:00.544 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:00.542+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:01.098 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:01.095+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:01.507 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:01.505+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:02.141 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:02.139+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:02.481 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:02.479+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:03.159 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:03.157+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:03.523 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:03.521+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:04.110 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:04.108+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:04.569 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:04.567+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:05.091 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:05.088+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:05.556 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:05.554+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:06.076 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:06.073+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:06.559 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:06.557+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:07.092 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:07.090+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:07.605 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:07.603+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:08.141 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:08.139+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:08.626 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:08.624+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:09.146 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:09.143+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:09.612 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:09.610+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:10.131 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:10.129+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:10.619 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:10.617+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:11.098 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:11.096+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:11.601 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:11.599+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:12.076 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:12.074+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:12.639 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:12.637+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:13.044 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:13.042+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:13.596 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:13.594+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:14.094 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:14.092+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:14.641 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:14.639+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:15.105 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:15.103+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:15.640 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:15.638+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:16.097 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:16.095+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:16.690 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:16.688+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:17.139 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:17.137+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:17.667 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:17.665+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:18.116 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:18.114+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:18.718 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:18.716+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:19.155 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:19.153+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:19.700 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:19.698+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:20.174 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:20.172+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:20.727 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:20.725+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:21.170 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:21.168+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:21.680 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:21.678+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:22.159 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:22.157+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:22.709 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:22.707+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:23.132 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:23.130+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:23.668 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:23.667+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:24.117 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:24.115+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:24.698 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:24.696+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:25.164 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:25.162+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:25.733 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:25.731+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:26.160 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:26.159+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:26.749 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:26.748+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:27.196 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:27.195+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:27.794 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:27.793+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:28.217 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:28.216+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:28.755 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:28.754+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:29.226 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:29.224+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:29.709 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:29.708+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:30.224 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:30.223+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:30.691 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:30.690+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:31.213 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:31.212+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:31.648 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:31.647+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:32.239 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:32.238+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:32.601 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:32.600+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:33.249 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:33.248+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:33.636 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:33.635+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:34.264 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:34.263+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:34.676 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:34.675+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:35.248 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:35.246+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:35.654 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:35.653+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:36.209 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:36.208+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:36.631 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:36.630+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:37.209 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:37.208+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:37.626 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:37.625+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:38.193 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:38.192+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:38.586 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:38.585+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:39.170 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:39.169+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:39.631 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:39.629+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:40.176 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:40.175+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:40.606 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:40.605+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:41.207 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:41.206+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:41.569 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:41.568+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:42.161 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:42.160+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:42.534 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:42.533+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:43.144 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:43.143+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:43.525 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:43.524+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:44.115 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:44.114+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:44.550 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:44.549+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:45.117 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:45.116+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:45.551 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:45.550+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:46.166 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:46.164+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:46.574 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:46.572+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:47.156 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:47.154+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:47.565 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:47.564+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:48.174 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:48.172+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:48.558 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:48.556+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:49.163 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:49.161+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:49.573 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:49.570+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:50.192 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:50.190+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:50.558 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:50.556+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:51.238 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:51.236+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:51.525 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:51.523+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:52.257 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:52.255+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:52.562 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:52.560+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:53.248 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:53.246+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:53.593 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:53.591+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:54.248 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:54.246+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:54.626 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:54.624+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:55.253 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:55.251+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:55.646 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:55.644+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:56.270 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:56.268+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:56.641 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:56.639+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:57.301 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:57.298+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:57.603 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:57.601+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:58.303 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:58.301+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:58.584 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:58.582+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:59.304 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:33:59.302+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:33:59.610 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:33:59.608+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:00.256 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:00.254+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:00.626 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:00.624+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:01.271 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:01.268+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:01.618 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:01.616+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:02.268 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:02.266+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:02.570 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:02.568+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:03.309 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:03.307+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:03.569 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:03.567+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:04.330 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:04.328+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:04.543 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:04.540+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:05.284 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:05.282+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:05.568 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:05.566+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:06.247 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:06.245+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:06.582 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:06.580+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:07.226 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:07.224+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:07.578 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:07.576+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:08.234 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:08.232+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:08.563 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:08.561+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:09.260 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:09.258+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:09.611 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:09.609+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:10.241 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:10.239+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:10.566 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:10.563+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:11.263 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:11.261+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:11.596 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:11.594+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:12.242 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:12.240+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:12.549 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:12.547+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:13.246 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:13.244+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:13.570 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:13.568+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:14.289 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:14.288+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:14.539 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:14.537+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:15.266 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:15.264+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:15.532 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:15.530+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:16.262 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:16.260+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:16.579 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:16.577+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:17.294 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:17.292+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:17.533 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:17.531+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:18.250 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:18.248+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:18.484 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:18.482+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:19.280 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:19.278+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:19.485 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:19.483+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:20.297 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:20.295+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:20.511 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:20.509+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:21.279 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:21.277+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:21.534 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:21.532+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:22.255 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:22.253+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:22.569 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:22.568+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:23.291 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:23.288+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:23.600 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:23.598+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:24.282 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:24.280+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:24.594 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:24.593+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:25.299 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:25.298+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:25.621 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:25.620+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:26.341 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:26.340+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:26.666 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:26.665+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:27.357 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:27.356+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:27.633 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:27.631+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:28.367 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:28.366+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:28.635 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:28.634+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:29.354 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:29.353+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:29.665 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:29.663+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:30.326 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:30.325+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:30.677 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:30.676+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:31.353 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:31.352+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:31.644 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:31.643+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:32.310 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:32.309+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:32.639 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:32.638+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:33.266 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:33.265+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:33.621 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:33.620+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:34.241 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:34.240+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:34.653 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:34.652+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:35.255 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:35.254+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:35.604 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:35.603+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:36.213 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:36.212+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:36.611 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:36.610+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:37.237 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:37.236+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:37.608 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:37.607+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:38.233 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:38.232+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:38.641 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:38.640+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:39.225 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:39.224+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:39.685 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:39.684+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:40.215 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:40.214+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:40.731 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:40.730+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:41.171 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:41.170+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:41.747 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:41.746+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:42.149 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:42.148+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:42.710 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:42.709+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:43.111 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:43.110+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:43.698 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:43.697+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:44.121 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:44.120+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:44.650 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:44.648+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:45.076 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:45.074+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:45.629 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:45.627+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:46.105 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:46.103+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:46.625 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:46.623+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:47.114 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:47.112+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:47.625 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:47.623+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:48.134 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:48.132+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:48.621 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:48.619+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:49.153 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:49.151+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:49.654 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:49.652+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:50.189 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:50.187+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:50.626 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:50.624+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:51.239 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:51.237+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:51.644 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:51.642+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:52.228 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:52.226+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:52.687 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:52.685+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:53.210 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:53.208+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:53.677 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:53.675+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:54.168 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:54.165+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:54.695 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:54.693+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:55.119 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:55.117+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:55.707 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:55.705+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:56.130 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:56.128+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:56.691 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:56.689+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:57.083 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:57.081+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:57.685 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:57.683+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:58.059 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:58.057+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:58.713 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:58.711+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:59.037 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:34:59.035+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:34:59.754 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:34:59.752+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:00.034 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:00.032+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:00.772 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:00.770+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:01.079 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:01.077+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:01.818 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:01.816+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:02.037 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:02.034+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:02.782 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:02.780+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:03.079 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:03.077+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:03.780 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:03.778+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:04.065 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:04.063+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:04.795 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:04.793+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:05.052 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:05.050+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:05.799 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:05.797+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:06.095 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:06.093+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:06.782 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:06.780+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:07.049 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:07.047+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:07.748 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:07.746+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:08.071 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:08.069+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:08.743 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:08.741+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:09.095 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:09.093+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:09.712 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:09.710+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:10.091 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:10.089+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:10.706 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:10.704+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:11.140 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:11.138+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:11.706 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:11.704+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:12.145 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:12.143+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:12.701 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:12.699+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:13.107 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:13.105+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:13.679 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:13.677+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:14.061 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:14.059+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:14.669 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:14.667+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:15.072 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:15.070+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:15.716 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:15.714+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:16.036 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:16.034+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:16.760 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:16.758+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:17.027 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:17.025+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:17.737 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:17.735+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:18.057 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:18.055+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:18.694 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:18.693+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:19.008 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:19.007+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:19.695 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:19.693+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:20.045 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:20.043+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:20.716 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:20.714+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:21.004 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:21.002+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:21.740 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:21.738+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:21.985 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:21.983+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:22.712 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:22.711+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:22.943 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:22.942+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:23.711 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:23.710+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:23.928 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:23.927+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:24.748 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:24.747+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:24.971 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:24.970+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:25.780 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:25.779+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:25.987 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:25.986+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:26.746 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:26.745+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:26.952 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:26.950+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:27.710 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:27.709+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:27.978 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:27.977+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:28.742 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:28.741+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:28.969 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:28.968+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:29.756 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:29.755+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:29.978 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:29.977+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:30.806 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:30.805+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:30.998 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:30.996+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:31.798 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:31.797+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:31.980 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:31.979+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:32.772 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:32.771+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:32.947 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:32.945+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:33.788 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:33.787+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:33.965 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:33.964+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:34.812 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:34.811+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:34.996 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:34.995+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:35.792 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:35.791+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:35.973 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:35.972+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:36.814 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:36.813+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:36.946 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:36.944+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:37.851 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:37.850+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:37.949 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:37.947+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:38.808 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:38.807+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:38.933 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:38.932+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:39.823 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:39.822+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:39.962 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:39.961+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:40.824 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:40.823+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:40.986 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:40.985+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:41.797 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:41.796+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:41.993 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:41.992+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:42.814 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:42.813+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:42.954 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:42.952+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:43.861 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:43.859+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:43.941 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:43.939+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:44.813 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:44.811+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:44.960 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:44.958+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:45.773 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:45.771+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:45.949 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:45.947+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:46.804 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:46.802+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:46.910 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:46.908+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:47.783 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:47.781+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:47.904 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:47.902+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:48.786 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:48.784+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:48.923 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:48.921+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:49.832 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:49.830+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:49.909 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:49.907+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:50.791 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:50.789+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:50.908 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:50.906+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:51.804 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:51.802+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:51.931 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:51.929+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:52.780 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:52.778+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:52.966 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:52.964+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:53.772 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:53.770+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:53.983 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:53.981+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:54.797 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:54.795+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:55.033 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:55.031+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:55.774 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:55.772+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:56.045 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:56.043+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:56.770 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:56.768+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:57.008 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:57.006+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:57.760 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:57.758+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:57.982 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:57.979+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:58.750 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:58.748+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:59.011 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:59.009+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:59.752 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:35:59.749+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:35:59.970 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:35:59.968+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:00.778 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:00.776+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:00.932 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:00.930+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:01.766 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:01.764+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:01.902 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:01.901+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:02.724 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:02.722+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:02.856 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:02.854+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:03.730 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:03.728+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:03.829 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:03.827+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:04.754 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:04.752+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:04.804 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:04.802+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:05.720 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:05.718+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:05.822 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:05.820+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:06.687 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:06.685+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:06.856 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:06.854+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:07.691 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:07.689+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:07.884 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:07.881+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:08.709 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:08.707+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:08.864 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:08.862+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:09.720 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:09.718+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:09.832 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:09.830+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:10.672 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:10.670+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:10.825 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:10.823+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:11.712 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:11.710+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:11.816 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:11.814+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:12.731 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:12.729+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:12.777 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:12.775+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:13.746 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:13.744+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:13.800 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:13.798+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:14.785 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:14.783+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:14.800 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:14.798+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:15.766 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:15.764+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:15.779 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:15.778+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:16.737 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:16.733+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:16.818 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:16.816+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:17.770 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:17.768+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:17.851 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:17.849+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:18.755 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:18.753+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:18.897 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:18.895+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:19.775 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:19.773+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:19.904 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:19.902+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:20.808 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:20.807+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:20.860 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:20.859+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:21.819 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:21.818+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:21.830 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:21.829+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:22.853 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:22.852+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:22.864 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:22.863+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:23.892 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:23.891+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:23.899 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:23.898+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:24.930 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:24.928+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:24.941 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:24.940+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:25.893 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:25.891+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:25.985 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:25.984+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:26.858 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:26.857+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:26.941 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:26.939+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:27.878 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:27.877+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:27.935 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:27.934+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:28.909 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:28.908+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:28.977 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:28.976+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:29.933 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:29.932+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:29.970 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:29.969+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:30.933 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:30.932+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:30.967 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:30.966+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:31.947 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:31.946+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:31.982 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:31.980+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:32.950 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:32.949+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:32.951 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:32.950+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:33.964 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:33.963+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:34.001 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:33.999+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:34.976 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:34.975+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:35.015 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:35.014+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:35.964 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:35.963+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:35.972 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:35.971+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:36.933 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:36.932+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:36.966 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:36.965+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:37.945 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:37.944+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:37.983 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:37.982+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:38.957 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:38.955+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:38.995 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:38.993+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:40.004 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:40.003+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:40.041 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:40.040+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:41.053 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:41.052+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:41.084 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:41.083+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:42.037 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:42.035+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:42.050 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:42.048+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:43.086 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:43.084+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:43.100 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:43.098+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:44.090 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:44.088+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:44.112 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:44.110+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:45.065 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:45.063+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:45.096 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:45.094+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:46.061 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:46.058+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:46.069 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:46.066+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:47.063 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:47.061+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:47.072 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:47.070+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:48.026 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:48.024+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:48.040 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:48.038+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:49.004 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:49.002+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:49.029 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:49.027+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:49.957 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:49.955+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:50.070 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:50.068+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:51.008 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:51.006+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:51.078 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:51.076+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:51.999 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:51.997+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:52.069 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:52.067+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:53.008 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:53.006+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:53.100 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:53.098+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:53.982 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:53.980+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:54.083 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:54.081+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:54.945 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:54.943+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:55.035 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:55.033+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:55.923 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:55.921+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:56.061 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:56.059+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:56.915 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:56.913+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:57.073 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:57.071+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:57.908 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:57.906+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:58.053 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:58.051+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:58.906 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:58.904+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:59.087 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:36:59.085+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:36:59.890 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:36:59.888+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:00.054 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:00.052+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:00.866 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:00.864+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:01.076 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:01.074+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:01.823 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:01.821+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:02.086 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:02.084+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:02.820 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:02.818+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:03.052 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:03.050+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:03.837 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:03.835+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:04.046 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:04.044+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:04.807 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:04.805+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:05.002 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:05.000+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:05.823 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:05.821+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:05.984 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:05.982+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:06.860 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:06.858+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:07.022 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:07.020+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:07.889 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:07.887+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:08.031 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:08.029+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:08.860 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:08.858+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:09.046 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:09.044+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:09.907 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:09.905+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:10.010 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:10.008+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:10.885 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:10.883+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:11.017 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:11.015+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:11.924 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:11.922+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:12.008 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:12.005+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:12.969 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:12.967+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:13.006 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:13.004+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:13.971 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:13.969+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:13.982 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:13.980+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:14.946 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:14.944+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:14.978 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:14.976+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:15.978 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:15.976+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:16.002 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:16.000+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:16.985 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:16.983+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:16.999 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:16.997+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:17.987 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:17.985+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:18.032 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:18.030+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:18.961 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:18.959+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:19.062 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:19.060+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:19.960 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:19.959+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:20.049 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:20.048+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:20.930 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:20.929+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:21.056 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:21.054+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:21.896 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:21.895+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:22.045 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:22.044+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:22.940 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:22.939+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:23.036 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:23.035+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:23.915 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:23.914+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:24.013 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:24.012+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:24.948 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:24.947+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:24.973 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:24.972+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:25.971 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:25.970+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:25.986 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:25.985+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:26.927 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:26.926+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:26.947 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:26.946+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:27.918 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:27.917+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:27.924 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:27.923+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:28.878 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:28.877+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:28.921 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:28.920+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:29.849 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:29.848+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:29.944 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:29.943+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:30.809 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:30.808+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:30.912 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:30.911+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:31.770 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:31.769+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:31.934 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:31.933+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:32.754 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:32.753+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:32.982 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:32.981+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:33.800 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:33.798+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:33.980 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:33.979+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:34.831 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:34.830+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:34.965 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:34.964+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:35.876 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:35.875+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:35.924 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:35.923+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:36.873 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:36.872+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:36.880 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:36.879+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:37.843 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:37.842+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:37.865 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:37.864+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:38.860 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:38.859+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:38.862 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:38.861+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:39.838 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:39.837+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:39.895 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:39.894+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:40.861 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:40.859+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:40.940 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:40.938+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:41.864 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:41.862+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:41.980 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:41.978+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:42.903 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:42.901+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:42.935 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:42.933+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:43.953 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:43.951+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:43.957 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:43.955+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:44.934 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:44.931+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:44.975 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:44.973+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:45.903 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:45.901+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:45.938 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:45.936+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:46.894 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:46.892+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:46.918 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:46.916+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:47.855 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:47.853+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:47.872 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:47.870+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:48.817 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:48.815+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:48.844 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:48.842+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:49.795 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:49.793+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:49.878 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:49.876+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:50.791 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:50.789+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:50.832 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:50.830+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:51.794 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:51.792+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:51.819 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:51.817+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:52.766 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:52.764+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:52.807 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:52.806+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:53.738 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:53.736+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:53.781 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:53.779+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:54.720 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:54.718+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:54.754 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:54.752+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:55.765 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:55.763+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:55.797 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:55.795+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:56.790 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:56.788+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:56.832 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:56.830+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:57.776 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:57.774+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:57.805 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:57.803+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:58.754 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:58.752+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:58.781 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:58.779+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:59.711 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:37:59.709+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:37:59.804 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:37:59.802+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:00.720 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:00.718+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:00.849 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:00.847+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:01.768 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:01.766+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:01.896 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:01.894+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:02.779 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:02.777+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:02.879 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:02.877+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:03.810 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:03.808+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:03.832 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:03.830+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:04.795 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:04.793+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:04.798 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:04.796+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:05.774 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:05.772+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:05.781 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:05.779+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:06.792 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:06.790+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:06.809 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:06.807+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:07.805 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:07.803+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:07.806 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:07.804+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:08.763 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:08.761+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:08.792 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:08.790+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:09.774 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:09.772+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:09.837 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:09.835+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:10.802 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:10.800+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:10.876 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:10.874+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:11.756 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:11.753+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:11.862 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:11.860+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:12.762 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:12.760+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:12.834 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:12.832+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:13.791 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:13.789+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:13.800 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:13.798+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:14.764 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:14.762+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:14.783 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:14.781+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:15.722 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:15.720+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:15.829 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:15.827+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:16.704 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:16.702+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:16.802 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:16.800+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:17.660 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:17.658+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:17.755 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:17.753+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:18.671 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:18.670+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:18.726 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:18.724+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:19.707 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:19.706+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:19.732 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:19.731+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:20.677 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:20.676+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:20.714 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:20.714+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:21.635 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:21.634+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:21.721 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:21.720+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:22.656 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:22.654+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:22.719 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:22.718+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:23.695 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:23.694+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:23.715 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:23.714+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:24.655 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:24.654+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:24.740 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:24.739+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:25.667 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:25.666+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:25.718 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:25.717+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:26.684 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:26.683+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:26.708 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:26.707+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:27.728 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:27.727+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:27.756 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:27.755+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:28.700 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:28.699+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:28.751 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:28.750+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:29.692 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:29.691+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:29.788 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:29.788+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:30.714 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:30.713+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:30.817 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:30.816+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:31.737 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:31.736+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:31.842 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:31.841+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:32.739 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:32.738+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:32.860 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:32.859+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:33.773 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:33.771+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:33.877 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:33.876+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:34.815 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:34.814+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:34.862 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:34.861+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:35.824 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:35.823+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:35.857 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:35.856+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:36.786 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:36.785+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:36.842 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:36.841+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:37.776 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:37.775+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:37.868 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:37.867+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:38.789 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:38.788+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:38.893 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:38.891+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:39.767 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:39.765+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:39.903 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:39.901+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:40.737 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:40.735+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:40.927 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:40.925+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:41.725 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:41.723+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:41.933 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:41.931+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:42.702 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:42.699+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:42.961 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:42.959+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:43.751 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:43.749+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:43.979 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:43.977+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:44.722 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:44.720+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:44.969 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:44.967+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:45.702 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:45.700+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:45.995 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:45.993+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:46.655 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:46.653+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:46.951 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:46.949+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:47.608 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:47.606+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:47.911 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:47.909+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:48.610 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:48.608+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:48.957 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:48.955+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:49.563 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:49.561+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:49.979 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:49.977+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:50.586 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:50.584+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:50.954 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:50.952+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:51.577 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:51.575+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:51.929 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:51.926+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:52.577 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:52.575+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:52.933 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:52.931+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:53.624 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:53.622+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:53.971 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:53.969+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:54.637 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:54.635+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:54.931 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:54.928+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:55.621 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:55.619+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:55.953 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:55.951+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:56.663 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:56.661+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:56.911 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:56.909+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:57.682 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:57.680+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:57.906 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:57.904+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:58.707 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:58.704+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:58.948 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:58.945+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:59.746 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:38:59.744+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:38:59.935 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:38:59.933+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:00.759 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:00.757+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:00.965 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:00.963+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:01.712 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:01.710+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:01.968 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:01.965+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:02.675 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:02.673+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:02.938 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:02.935+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:03.695 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:03.693+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:03.974 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:03.972+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:04.730 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:04.728+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:04.996 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:04.994+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:05.684 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:05.682+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:05.989 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:05.986+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:06.694 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:06.692+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:06.989 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:06.987+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:07.729 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:07.727+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:07.944 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:07.942+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:08.757 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:08.755+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:08.959 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:08.957+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:09.799 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:09.797+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:09.921 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:09.919+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:10.757 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:10.755+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:10.909 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:10.907+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:11.766 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:11.764+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:11.929 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:11.927+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:12.791 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:12.789+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:12.893 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:12.891+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:13.758 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:13.756+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:13.933 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:13.931+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:14.781 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:14.779+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:14.890 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:14.889+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:15.748 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:15.746+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:15.863 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:15.861+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:16.785 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:16.783+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:16.842 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:16.840+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:17.788 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:17.787+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:17.844 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:17.843+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:18.796 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:18.795+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:18.803 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:18.802+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:19.752 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:19.751+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:19.851 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:19.850+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:20.739 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:20.738+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:20.818 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:20.817+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:21.697 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:21.696+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:21.824 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:21.823+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:22.666 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:22.665+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:22.810 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:22.809+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:23.622 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:23.621+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:23.786 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:23.785+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:24.573 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:24.572+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:24.828 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:24.827+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:25.607 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:25.606+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:25.782 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:25.781+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:26.650 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:26.649+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:26.798 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:26.797+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:27.663 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:27.662+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:27.800 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:27.799+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:28.646 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:28.645+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:28.812 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:28.811+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:29.658 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:29.657+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:29.806 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:29.805+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:30.669 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:30.668+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:30.837 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:30.836+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:31.645 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:31.644+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:31.888 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:31.886+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:32.664 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:32.663+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:32.900 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:32.899+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:33.640 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:33.639+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:33.911 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:33.910+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:34.664 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:34.663+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:34.898 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:34.897+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:35.691 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:35.690+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:35.896 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:35.895+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:36.734 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:36.733+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:36.916 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:36.915+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:37.771 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:37.769+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:37.944 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:37.942+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:38.788 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:38.786+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:38.977 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:38.974+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:39.760 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:39.758+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:40.020 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:40.018+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:40.724 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:40.722+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:41.026 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:41.024+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:41.749 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:41.747+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:42.031 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:42.029+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:42.747 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:42.745+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:43.005 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:43.003+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:43.771 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:43.769+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:43.991 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:43.989+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:44.767 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:44.765+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:44.962 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:44.960+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:45.814 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:45.812+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:45.943 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:45.941+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:46.853 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:46.851+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:46.894 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:46.892+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:47.853 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:47.851+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:47.902 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:47.900+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:48.836 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:48.834+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:48.952 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:48.950+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:49.806 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:49.804+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:49.915 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:49.913+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:50.818 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:50.815+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:50.938 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:50.936+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:51.864 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:51.862+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:51.979 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:51.977+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:52.874 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:52.872+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:52.972 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:52.970+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:53.910 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:53.908+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:54.017 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:54.015+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:54.922 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:54.920+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:55.030 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:55.028+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:55.876 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:55.874+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:55.999 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:55.997+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:56.924 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:56.922+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:56.996 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:56.994+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:57.909 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:57.907+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:58.002 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:58.000+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:58.959 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:58.957+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:59.030 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:39:59.029+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:39:59.942 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:39:59.940+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:00.002 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:00.000+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:00.990 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:00.988+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:01.004 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:01.002+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:02.029 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:02.027+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:02.044 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:02.042+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:02.997 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:02.996+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:03.055 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:03.053+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:04.001 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:03.998+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:04.028 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:04.026+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:04.970 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:04.968+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:05.074 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:05.072+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:05.946 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:05.944+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:06.112 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:06.109+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:06.924 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:06.922+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:07.086 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:07.084+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:07.889 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:07.887+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:08.132 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:08.130+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:08.878 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:08.876+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:09.102 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:09.100+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:09.915 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:09.912+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:10.091 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:10.088+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:10.876 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:10.873+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:11.075 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:11.073+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:11.832 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:11.830+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:12.060 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:12.058+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:12.822 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:12.820+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:13.026 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:13.024+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:13.782 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:13.780+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:14.035 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:14.033+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:14.738 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:14.737+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:15.071 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:15.070+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:15.756 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:15.755+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:16.060 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:16.059+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:16.766 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:16.765+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:17.110 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:17.109+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:17.773 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:17.772+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:18.081 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:18.080+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:18.778 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:18.777+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:19.115 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:19.114+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:19.789 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:19.788+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:20.128 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:20.126+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:20.743 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:20.742+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:21.160 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:21.159+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:21.742 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:21.741+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:22.153 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:22.152+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:22.728 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:22.727+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:23.196 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:23.195+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:23.742 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:23.741+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:24.150 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:24.149+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:24.789 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:24.788+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:25.189 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:25.188+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:25.837 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:25.836+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:26.219 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:26.218+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:26.828 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:26.827+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:27.211 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:27.210+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:27.784 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:27.783+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:28.258 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:28.257+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:28.818 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:28.817+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:29.233 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:29.232+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:29.789 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:29.788+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:30.273 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:30.272+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:30.816 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:30.815+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:31.308 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:31.307+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:31.858 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:31.857+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:32.262 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:32.261+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:32.867 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:32.866+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:33.218 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:33.217+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:33.901 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:33.900+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:34.268 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:34.267+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:34.875 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:34.874+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:35.282 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:35.281+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:35.922 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:35.920+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:36.249 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:36.247+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:36.881 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:36.879+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:37.233 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:37.231+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:37.848 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:37.846+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:38.246 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:38.244+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:38.810 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:38.808+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:39.218 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:39.216+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:39.799 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:39.797+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:40.216 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:40.214+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:40.763 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:40.761+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:41.258 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:41.256+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:41.744 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:41.742+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:42.246 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:42.244+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:42.779 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:42.777+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:43.211 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:43.209+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:43.736 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:43.734+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:44.227 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:44.225+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:44.776 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:44.774+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:45.208 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:45.206+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:45.728 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:45.726+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:46.184 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:46.182+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:46.702 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:46.700+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:47.229 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:47.227+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:47.682 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:47.680+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:48.232 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:48.230+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:48.728 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:48.726+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:49.232 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:49.230+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:49.743 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:49.741+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:50.198 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:50.196+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:50.777 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:50.774+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:51.175 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:51.173+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:51.744 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:51.742+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:52.213 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:52.211+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:52.727 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:52.724+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:53.167 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:53.165+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:53.760 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:53.753+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:54.154 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:54.152+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:54.706 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:54.704+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:55.174 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:55.172+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:55.685 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:55.683+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:56.215 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:56.213+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:56.669 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:56.667+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:57.242 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:57.240+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:57.666 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:57.664+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:58.265 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:58.263+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:58.622 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:58.620+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:59.312 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:40:59.310+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:40:59.672 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:40:59.669+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:00.274 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:00.271+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:00.631 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:00.629+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:01.246 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:01.244+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:01.618 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:01.616+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:02.228 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:02.226+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:02.591 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:02.589+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:03.212 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:03.210+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:03.606 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:03.604+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:04.198 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:04.196+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:04.624 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:04.622+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:05.243 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:05.241+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:05.601 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:05.598+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:06.198 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:06.195+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:06.602 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:06.600+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:07.192 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:07.190+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:07.624 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:07.622+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:08.217 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:08.215+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:08.585 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:08.583+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:09.252 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:09.250+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:09.542 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:09.540+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:10.252 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:10.250+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:10.511 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:10.508+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:11.227 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:11.225+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:11.473 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:11.470+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:12.203 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:12.201+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:12.489 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:12.488+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:13.210 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:13.208+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:13.478 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:13.476+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:14.173 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:14.172+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:14.519 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:14.518+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:15.179 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:15.178+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:15.524 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:15.523+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:16.223 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:16.221+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:16.487 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:16.486+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:17.186 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:17.185+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:17.492 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:17.491+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:18.178 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:18.177+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:18.489 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:18.488+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:19.173 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:19.172+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:19.519 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:19.518+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:20.218 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:20.217+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:20.512 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:20.511+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:21.219 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:21.218+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:21.547 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:21.546+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:22.239 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:22.237+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:22.523 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:22.522+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:23.228 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:23.225+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:23.574 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:23.572+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:24.223 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:24.218+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:24.582 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:24.581+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:25.218 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:25.217+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:25.588 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:25.587+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:26.177 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:26.176+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:26.616 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:26.615+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:27.190 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:27.189+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:27.585 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:27.584+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:28.207 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:28.206+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:28.629 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:28.628+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:29.227 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:29.226+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:29.624 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:29.623+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:30.267 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:30.266+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:30.583 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:30.582+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:31.226 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:31.225+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:31.551 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:31.550+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:32.246 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:32.245+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:32.553 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:32.552+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:33.239 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:33.238+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:33.542 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:33.541+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:34.283 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:34.281+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:34.538 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:34.536+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:35.254 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:35.252+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:35.580 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:35.578+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:36.305 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:36.302+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:36.604 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:36.602+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:37.354 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:37.352+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:37.652 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:37.650+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:38.342 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:38.340+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:38.698 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:38.696+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:39.321 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:39.318+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:39.736 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:39.733+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:40.303 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:40.301+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:40.751 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:40.748+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:41.319 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:41.317+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:41.773 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:41.771+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:42.276 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:42.274+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:42.812 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:42.809+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:43.285 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:43.283+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:43.813 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:43.811+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:44.251 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:44.249+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:44.790 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:44.788+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:45.266 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:45.264+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:45.815 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:45.812+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:46.245 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:46.243+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:46.804 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:46.802+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:47.205 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:47.203+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:47.794 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:47.792+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:48.233 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:48.231+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:48.797 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:48.795+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:49.278 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:49.276+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:49.839 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:49.837+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:50.281 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:50.279+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:50.837 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:50.835+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:51.289 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:51.287+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:51.802 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:51.800+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:52.263 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:52.261+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:52.772 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:52.770+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:53.266 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:53.264+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:53.769 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:53.767+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:54.306 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:54.304+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:54.744 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:54.742+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:55.275 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:55.273+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:55.761 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:55.759+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:56.324 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:56.322+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:56.796 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:56.794+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:57.347 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:57.345+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:57.808 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:57.805+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:58.391 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:58.389+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:58.825 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:58.823+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:59.440 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:41:59.438+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:41:59.804 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:41:59.802+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:00.402 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:00.400+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:00.836 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:00.834+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:01.396 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:01.394+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:01.827 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:01.825+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:02.391 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:02.389+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:02.802 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:02.800+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:03.363 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:03.361+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:03.836 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:03.834+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:04.332 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:04.330+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:04.886 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:04.884+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:05.298 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:05.296+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:05.885 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:05.882+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:06.310 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:06.308+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:06.932 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:06.930+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:07.332 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:07.330+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:07.936 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:07.934+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:08.366 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:08.364+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:08.966 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:08.964+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:09.368 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:09.366+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:09.932 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:09.930+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:10.412 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:10.410+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:10.884 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:10.882+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:11.416 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:11.414+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:11.906 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:11.904+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:12.426 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:12.425+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:12.861 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:12.860+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:13.440 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:13.439+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:13.829 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:13.827+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:14.438 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:14.437+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:14.870 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:14.869+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:15.434 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:15.433+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:15.858 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:15.857+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:16.430 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:16.429+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:16.858 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:16.857+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:17.425 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:17.424+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:17.848 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:17.847+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:18.455 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:18.454+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:18.837 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:18.836+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:19.465 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:19.464+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:19.804 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:19.803+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:20.470 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:20.469+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:20.770 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:20.769+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:21.461 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:21.460+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:21.768 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:21.767+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:22.426 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:22.425+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:22.756 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:22.755+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:23.392 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:23.391+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:23.790 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:23.789+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:24.353 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:24.352+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:24.838 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:24.837+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:25.345 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:25.344+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:25.859 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:25.858+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:26.350 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:26.349+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:26.812 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:26.811+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:27.376 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:27.375+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:27.850 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:27.849+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:28.353 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:28.352+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:28.809 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:28.808+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:29.327 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:29.326+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:29.766 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:29.765+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:30.334 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:30.333+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:30.738 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:30.737+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:31.356 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:31.355+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:31.718 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:31.717+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:32.360 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:32.358+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:32.700 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:32.698+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:33.394 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:33.392+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:33.657 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:33.655+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:34.404 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:34.402+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:34.670 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:34.668+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:35.368 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:35.366+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:35.682 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:35.679+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:36.386 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:36.384+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:36.702 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:36.700+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:37.360 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:37.358+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:37.722 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:37.720+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:38.374 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:38.372+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:38.737 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:38.736+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:39.335 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:39.333+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:39.769 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:39.767+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:40.318 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:40.316+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:40.805 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:40.803+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:41.316 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:41.314+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:41.816 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:41.814+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:42.314 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:42.312+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:42.840 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:42.838+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:43.294 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:43.292+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:43.805 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:43.803+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:44.310 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:44.308+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:44.784 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:44.782+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:45.266 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:45.264+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:45.813 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:45.811+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:46.245 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:46.243+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:46.822 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:46.820+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:47.261 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:47.259+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:47.791 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:47.789+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:48.231 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:48.230+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:48.792 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:48.790+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:49.223 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:49.221+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:49.785 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:49.783+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:50.235 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:50.233+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:50.820 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:50.818+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:51.214 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:51.212+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:51.856 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:51.854+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:52.168 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:52.166+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:52.825 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:52.823+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:53.132 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:53.130+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:53.816 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:53.814+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:54.146 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:54.143+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:54.834 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:54.832+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:55.156 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:55.154+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:55.831 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:55.829+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:56.136 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:56.134+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:56.818 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:56.815+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:57.174 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:57.172+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:57.858 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:57.855+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:58.138 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:58.136+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:58.826 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:58.824+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:59.110 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:42:59.108+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:42:59.784 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:42:59.781+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:00.154 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:00.152+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:00.774 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:00.772+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:01.180 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:01.178+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:01.772 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:01.770+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:02.223 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:02.221+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:02.777 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:02.774+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:03.229 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:03.227+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:03.799 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:03.797+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:04.278 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:04.276+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:04.825 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:04.823+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:05.328 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:05.326+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:05.851 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:05.849+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:06.330 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:06.328+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:06.901 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:06.899+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:07.314 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:07.312+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:07.906 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:07.904+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:08.315 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:08.313+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:08.905 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:08.904+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:09.341 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:09.339+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:09.905 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:09.903+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:10.351 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:10.350+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:10.925 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:10.924+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:11.339 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:11.338+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:11.933 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:11.932+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:12.350 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:12.349+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:12.978 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:12.977+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:13.323 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:13.322+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:13.963 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:13.961+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:14.283 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:14.282+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:14.937 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:14.936+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:15.295 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:15.294+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:15.962 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:15.961+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:16.327 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:16.326+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:16.944 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:16.943+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:17.349 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:17.348+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:17.983 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:17.982+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:18.372 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:18.371+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:18.951 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:18.950+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:19.405 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:19.404+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:19.989 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:19.987+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:20.390 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:20.389+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:21.005 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:21.004+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:21.389 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:21.388+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:21.973 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:21.972+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:22.370 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:22.369+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:23.007 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:23.006+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:23.365 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:23.364+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:23.983 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:23.982+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:24.350 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:24.349+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:25.009 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:25.008+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:25.386 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:25.385+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:25.978 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:25.977+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:26.351 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:26.350+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:26.947 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:26.946+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:27.303 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:27.301+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:27.997 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:27.996+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:28.334 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:28.333+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:29.007 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:29.006+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:29.333 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:29.332+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:30.042 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:30.041+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:30.300 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:30.298+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:31.073 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:31.071+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:31.331 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:31.329+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:32.119 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:32.117+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:32.351 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:32.348+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:33.096 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:33.094+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:33.378 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:33.376+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:34.063 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:34.061+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:34.335 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:34.333+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:35.015 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:35.013+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:35.377 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:35.375+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:35.987 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:35.985+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:36.365 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:36.363+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:37.005 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:37.003+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:37.402 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:37.400+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:37.993 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:37.991+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:38.421 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:38.419+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:38.947 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:38.945+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:39.445 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:39.443+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:39.951 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:39.948+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:40.436 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:40.434+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:40.939 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:40.937+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:41.389 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:41.387+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:41.904 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:41.902+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:42.392 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:42.389+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:42.909 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:42.907+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:43.436 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:43.434+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:43.865 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:43.862+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:44.388 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:44.386+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:44.867 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:44.865+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:45.385 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:45.383+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:45.881 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:45.879+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:46.363 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:46.361+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:46.879 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:46.876+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:47.351 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:47.349+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:47.905 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:47.903+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:48.383 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:48.381+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:48.875 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:48.873+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:49.365 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:49.363+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:49.919 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:49.917+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:50.380 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:50.378+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:50.922 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:50.920+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:51.371 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:51.369+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:51.889 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:51.887+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:52.378 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:52.375+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:52.871 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:52.869+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:53.342 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:53.339+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:53.902 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:53.900+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:54.347 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:54.345+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:54.888 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:54.886+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:55.361 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:55.359+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:55.911 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:55.909+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:56.321 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:56.319+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:56.882 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:56.880+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:57.342 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:57.340+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:57.849 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:57.847+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:58.377 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:58.375+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:58.857 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:58.855+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:59.342 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:43:59.340+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:43:59.816 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:43:59.814+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:00.335 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:00.333+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:00.809 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:00.807+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:01.310 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:01.308+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:01.827 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:01.825+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:02.286 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:02.284+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:02.797 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:02.795+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:03.298 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:03.296+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:03.754 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:03.752+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:04.311 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:04.309+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:04.783 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:04.781+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:05.313 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:05.311+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:05.813 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:05.811+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:06.289 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:06.287+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:06.796 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:06.794+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:07.302 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:07.300+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:07.777 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:07.775+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:08.269 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:08.267+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:08.821 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:08.820+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:09.234 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:09.233+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:09.838 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:09.837+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:10.239 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:10.238+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:10.825 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:10.824+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:11.227 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:11.227+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:11.846 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:11.845+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:12.249 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:12.248+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:12.889 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:12.887+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:13.264 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:13.263+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:13.926 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:13.925+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:14.294 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:14.293+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:14.940 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:14.939+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:15.345 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:15.343+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:15.894 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:15.893+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:16.328 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:16.327+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:16.924 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:16.923+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:17.303 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:17.302+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:17.926 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:17.925+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:18.257 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:18.256+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:18.899 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:18.898+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:19.258 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:19.257+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:19.882 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:19.881+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:20.252 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:20.251+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:20.844 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:20.843+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:21.254 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:21.253+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:21.874 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:21.873+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:22.247 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:22.246+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:22.916 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:22.915+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:23.205 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:23.204+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:23.888 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:23.887+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:24.212 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:24.211+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:24.938 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:24.937+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:25.203 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:25.202+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:25.890 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:25.889+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:26.174 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:26.173+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:26.914 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:26.913+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:27.215 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:27.214+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:27.917 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:27.916+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:28.217 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:28.216+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:28.902 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:28.900+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:29.189 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:29.187+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:29.918 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:29.916+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:30.174 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:30.172+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:30.934 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:30.932+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:31.222 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:31.220+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:31.983 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:31.981+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:32.264 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:32.262+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:32.956 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:32.954+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:33.312 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:33.309+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:33.938 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:33.936+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:34.282 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:34.280+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:34.930 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:34.928+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:35.242 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:35.239+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:35.934 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:35.932+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:36.194 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:36.192+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:36.918 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:36.916+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:37.210 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:37.208+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:37.897 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:37.895+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:38.165 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:38.163+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:38.894 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:38.892+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:39.176 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:39.174+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:39.886 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:39.884+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:40.174 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:40.172+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:40.876 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:40.874+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:41.165 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:41.163+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:41.904 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:41.902+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:42.173 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:42.171+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:42.937 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:42.935+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:43.163 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:43.161+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:43.987 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:43.985+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:44.120 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:44.118+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:44.999 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:44.997+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:45.136 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:45.134+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:45.953 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:45.951+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:46.142 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:46.140+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:46.947 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:46.945+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:47.097 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:47.095+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:47.927 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:47.925+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:48.080 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:48.078+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:48.897 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:48.894+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:49.087 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:49.085+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:49.876 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:49.874+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:50.090 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:50.088+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:50.886 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:50.884+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:51.077 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:51.075+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:51.864 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:51.862+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:52.126 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:52.124+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:52.848 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:52.846+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:53.088 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:53.086+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:53.863 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:53.861+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:54.115 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:54.113+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:54.890 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:54.888+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:55.154 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:55.152+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:55.859 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:55.857+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:56.172 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:56.170+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:56.817 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:56.815+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:57.140 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:57.138+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:57.844 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:57.842+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:58.180 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:58.178+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:58.833 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:58.831+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:59.217 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:44:59.215+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:44:59.843 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:44:59.841+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:00.195 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:00.193+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:00.879 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:00.877+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:01.200 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:01.198+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:01.848 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:01.846+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:02.232 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:02.230+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:02.897 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:02.896+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:03.260 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:03.258+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:03.888 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:03.886+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:04.293 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:04.291+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:04.841 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:04.839+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:05.330 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:05.328+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:05.823 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:05.821+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:06.281 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:06.280+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:06.869 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:06.868+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:07.297 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:07.296+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:07.884 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:07.883+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:08.249 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:08.248+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:08.919 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:08.918+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:09.286 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:09.285+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:09.915 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:09.914+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:10.328 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:10.327+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:10.964 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:10.963+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:11.331 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:11.330+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:11.979 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:11.978+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:12.319 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:12.318+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:12.931 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:12.930+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:13.319 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:13.318+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:13.926 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:13.925+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:14.360 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:14.359+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:14.895 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:14.894+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:15.354 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:15.353+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:15.896 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:15.895+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:16.332 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:16.331+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:16.914 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:16.913+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:17.382 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:17.380+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:17.875 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:17.874+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:18.383 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:18.382+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:18.903 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:18.901+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:19.355 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:19.354+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:19.883 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:19.882+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:20.368 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:20.367+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:20.862 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:20.861+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:21.324 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:21.323+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:21.894 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:21.893+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:22.316 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:22.315+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:22.903 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:22.902+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:23.281 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:23.280+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:23.881 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:23.879+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:24.322 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:24.321+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:24.910 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:24.909+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:25.347 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:25.346+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:25.905 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:25.904+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:26.375 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:26.374+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:26.915 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:26.913+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:27.357 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:27.355+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:27.928 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:27.926+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:28.323 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:28.321+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:28.918 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:28.916+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:29.363 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:29.361+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:29.883 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:29.881+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:30.340 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:30.338+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:30.911 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:30.909+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:31.331 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:31.329+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:31.922 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:31.920+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:32.304 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:32.303+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:32.942 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:32.940+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:33.263 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:33.261+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:33.945 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:33.943+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:34.277 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:34.275+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:34.918 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:34.916+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:35.266 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:35.264+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:35.899 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:35.897+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:36.263 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:36.261+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:36.903 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:36.901+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:37.217 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:37.215+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:37.918 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:37.916+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:38.203 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:38.201+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:38.956 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:38.954+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:39.169 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:39.167+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:39.982 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:39.980+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:40.121 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:40.119+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:40.980 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:40.978+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:41.155 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:41.153+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:41.953 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:41.948+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:42.143 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:42.141+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:42.905 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:42.903+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:43.191 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:43.189+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:43.877 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:43.875+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:44.185 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:44.183+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:44.876 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:44.874+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:45.227 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:45.225+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:45.913 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:45.911+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:46.235 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:46.233+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:46.932 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:46.930+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:47.195 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:47.192+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:47.891 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:47.889+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:48.177 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:48.175+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:48.855 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:48.853+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:49.154 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:49.152+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:49.837 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:49.835+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:50.126 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:50.124+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:50.876 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:50.874+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:51.138 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:51.136+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:51.841 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:51.839+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:52.184 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:52.182+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:52.829 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:52.827+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:53.176 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:53.174+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:53.877 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:53.875+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:54.201 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:54.199+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:54.843 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:54.841+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:55.241 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:55.239+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:55.807 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:55.805+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:56.217 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:56.215+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:56.762 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:56.760+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:57.228 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:57.226+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:57.809 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:57.807+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:58.264 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:58.262+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:58.772 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:58.770+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:59.299 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:45:59.297+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:45:59.726 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:45:59.724+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:00.287 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:00.285+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:00.757 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:00.755+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:01.306 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:01.304+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:01.750 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:01.748+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:02.335 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:02.333+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:02.788 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:02.786+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:03.305 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:03.303+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:03.756 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:03.754+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:04.308 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:04.306+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:04.710 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:04.708+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:05.335 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:05.334+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:05.710 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:05.709+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:06.352 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:06.351+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:06.746 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:06.745+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:07.303 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:07.302+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:07.774 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:07.773+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:08.317 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:08.316+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:08.738 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:08.737+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:09.306 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:09.305+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:09.789 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:09.788+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:10.303 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:10.302+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:10.832 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:10.831+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:11.349 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:11.348+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:11.856 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:11.855+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:12.305 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:12.304+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:12.847 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:12.845+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:13.262 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:13.261+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:13.834 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:13.833+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:14.310 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:14.308+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:14.849 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:14.848+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:15.342 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:15.341+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:15.843 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:15.842+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:16.359 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:16.358+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:16.870 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:16.869+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:17.316 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:17.315+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:17.852 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:17.851+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:18.290 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:18.289+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:18.839 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:18.838+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:19.304 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:19.303+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:19.806 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:19.805+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:20.345 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:20.344+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:20.836 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:20.835+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:21.352 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:21.350+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:21.818 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:21.816+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:22.394 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:22.393+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:22.854 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:22.852+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:23.416 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:23.415+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:23.874 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:23.873+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:24.374 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:24.373+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:24.831 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:24.830+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:25.424 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:25.422+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:25.856 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:25.854+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:26.406 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:26.403+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:26.842 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:26.840+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:27.377 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:27.375+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:27.832 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:27.830+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:28.391 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:28.389+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:28.824 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:28.822+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:29.359 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:29.357+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:29.866 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:29.864+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:30.344 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:30.342+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:30.903 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:30.901+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:31.368 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:31.366+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:31.932 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:31.930+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:32.417 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:32.415+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:32.927 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:32.925+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:33.435 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:33.433+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:33.917 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:33.915+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:34.461 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:34.459+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:34.968 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:34.966+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:35.472 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:35.470+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:36.018 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:36.016+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:36.446 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:36.444+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:37.020 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:37.018+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:37.434 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:37.432+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:38.070 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:38.068+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:38.477 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:38.476+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:39.112 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:39.110+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:39.518 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:39.516+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:40.066 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:40.064+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:40.476 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:40.474+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:41.086 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:41.084+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:41.480 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:41.478+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:42.079 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:42.077+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:42.472 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:42.470+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:43.057 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:43.055+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:43.438 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:43.436+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:44.082 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:44.080+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:44.456 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:44.454+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:45.086 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:45.084+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:45.466 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:45.464+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:46.087 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:46.085+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:46.482 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:46.480+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:47.062 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:47.060+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:47.491 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:47.489+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:48.066 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:48.064+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:48.518 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:48.516+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:49.042 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:49.040+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:49.492 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:49.490+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:50.053 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:50.051+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:50.472 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:50.470+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:51.071 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:51.069+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:51.460 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:51.458+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:52.073 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:52.071+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:52.414 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:52.412+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:53.036 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:53.033+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:53.387 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:53.385+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:54.052 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:54.050+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:54.436 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:54.434+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:55.101 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:55.099+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:55.438 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:55.436+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:56.063 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:56.061+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:56.476 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:56.474+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:57.025 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:57.023+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:57.479 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:57.477+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:57.982 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:57.980+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:58.514 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:58.512+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:59.001 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:46:58.999+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:46:59.534 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:46:59.532+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:00.012 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:00.010+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:00.551 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:00.549+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:00.993 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:00.991+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:01.562 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:01.560+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:02.034 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:02.032+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:02.538 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:02.536+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:02.987 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:02.986+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:03.586 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:03.585+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:04.024 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:04.023+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:04.551 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:04.550+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:04.980 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:04.979+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:05.529 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:05.528+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:05.959 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:05.958+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:06.545 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:06.544+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:06.952 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:06.951+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:07.530 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:07.529+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:08.000 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:07.999+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:08.554 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:08.553+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:09.048 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:09.047+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:09.593 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:09.592+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:10.086 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:10.085+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:10.553 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:10.552+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:11.094 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:11.093+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:11.579 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:11.578+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:12.093 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:12.092+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:12.586 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:12.585+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:13.052 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:13.051+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:13.625 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:13.623+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:14.078 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:14.077+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:14.648 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:14.647+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:15.074 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:15.073+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:15.643 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:15.642+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:16.032 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:16.031+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:16.689 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:16.688+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:17.039 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:17.038+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:17.651 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:17.650+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:18.060 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:18.059+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:18.640 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:18.639+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:19.022 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:19.021+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:19.675 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:19.674+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:20.060 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:20.059+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:20.668 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:20.666+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:21.034 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:21.033+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:21.704 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:21.703+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:22.074 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:22.073+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:22.674 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:22.673+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:23.095 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:23.094+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:23.632 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:23.630+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:24.066 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:24.064+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:24.601 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:24.599+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:25.062 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:25.060+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:25.635 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:25.633+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:26.028 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:26.026+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:26.634 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:26.632+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:26.991 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:26.990+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:27.602 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:27.600+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:27.973 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:27.971+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:28.564 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:28.562+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:28.989 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:28.987+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:29.542 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:29.539+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:30.016 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:30.014+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:30.538 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:30.536+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:30.989 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:30.987+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:31.585 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:31.583+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:31.991 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:31.989+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:32.571 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:32.569+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:33.027 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:33.025+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:33.613 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:33.611+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:34.038 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:34.036+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:34.611 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:34.609+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:35.070 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:35.068+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:35.596 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:35.594+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:36.099 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:36.097+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:36.582 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:36.579+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:37.064 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:37.062+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:37.615 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:37.612+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:38.073 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:38.071+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:38.644 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:38.642+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:39.091 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:39.088+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:39.667 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:39.665+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:40.140 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:40.138+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:40.618 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:40.616+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:41.179 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:41.177+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:41.650 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:41.648+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:42.161 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:42.159+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:42.652 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:42.650+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:43.118 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:43.116+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:43.624 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:43.622+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:44.116 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:44.114+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:44.672 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:44.670+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:45.150 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:45.148+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:45.720 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:45.718+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:46.116 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:46.114+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:46.710 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:46.708+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:47.094 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:47.092+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:47.689 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:47.687+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:48.125 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:48.123+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:48.699 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:48.697+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:49.104 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:49.102+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:49.676 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:49.674+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:50.113 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:50.112+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:50.645 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:50.642+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:51.143 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:51.141+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:51.597 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:51.595+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:52.152 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:52.150+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:52.594 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:52.592+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:53.194 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:53.192+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:53.556 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:53.554+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:54.202 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:54.200+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:54.549 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:54.547+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:55.242 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:55.240+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:55.543 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:55.540+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:56.259 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:56.256+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:56.518 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:56.516+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:57.232 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:57.230+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:57.554 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:57.551+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:58.226 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:58.224+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:58.559 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:58.558+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:59.214 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:47:59.211+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:47:59.526 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:47:59.524+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:00.252 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:00.250+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:00.522 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:00.520+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:01.297 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:01.295+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:01.566 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:01.565+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:02.260 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:02.259+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:02.601 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:02.600+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:03.285 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:03.284+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:03.635 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:03.634+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:04.260 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:04.259+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:04.619 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:04.618+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:05.258 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:05.257+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:05.590 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:05.589+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:06.227 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:06.226+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:06.627 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:06.626+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:07.213 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:07.211+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:07.640 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:07.639+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:08.236 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:08.235+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:08.676 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:08.675+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:09.250 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:09.249+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:09.680 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:09.679+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:10.228 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:10.227+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:10.660 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:10.659+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:11.186 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:11.185+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:11.656 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:11.655+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:12.220 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:12.219+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:12.616 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:12.615+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:13.192 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:13.191+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:13.571 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:13.570+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:14.174 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:14.173+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:14.620 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:14.619+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:15.141 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:15.140+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:15.621 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:15.620+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:16.182 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:16.181+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:16.628 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:16.627+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:17.140 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:17.139+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:17.677 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:17.676+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:18.160 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:18.159+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:18.642 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:18.640+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:19.150 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:19.149+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:19.630 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:19.629+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:20.102 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:20.101+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:20.589 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:20.588+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:21.053 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:21.052+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:21.584 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:21.583+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:22.099 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:22.097+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:22.592 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:22.590+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:23.059 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:23.057+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:23.620 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:23.618+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:24.042 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:24.040+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:24.577 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:24.575+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:25.052 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:25.050+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:25.536 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:25.534+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:26.037 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:26.035+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:26.544 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:26.542+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:27.048 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:27.046+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:27.550 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:27.548+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:28.027 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:28.025+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:28.507 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:28.505+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:29.056 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:29.054+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:29.482 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:29.480+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:30.021 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:30.019+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:30.499 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:30.497+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:31.037 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:31.035+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:31.514 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:31.512+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:31.999 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:31.997+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:32.506 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:32.504+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:33.017 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:33.015+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:33.532 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:33.530+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:34.061 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:34.059+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:34.485 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:34.483+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:35.036 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:35.034+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:35.471 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:35.469+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:36.066 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:36.064+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:36.516 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:36.514+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:37.074 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:37.072+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:37.488 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:37.486+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:38.118 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:38.116+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:38.454 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:38.452+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:39.151 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:39.149+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:39.478 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:39.476+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:40.146 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:40.143+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:40.484 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:40.482+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:41.181 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:41.178+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:41.530 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:41.528+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:42.190 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:42.188+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:42.492 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:42.489+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:43.208 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:43.206+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:43.494 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:43.492+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:44.175 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:44.173+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:44.524 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:44.522+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:45.200 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:45.198+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:45.544 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:45.542+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:46.168 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:46.166+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:46.518 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:46.515+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:47.186 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:47.184+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:47.472 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:47.470+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:48.208 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:48.206+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:48.504 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:48.502+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:49.196 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:49.194+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:49.513 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:49.511+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:50.215 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:50.213+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:50.511 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:50.509+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:51.167 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:51.165+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:51.481 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:51.479+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:52.132 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:52.130+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:52.528 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:52.526+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:53.125 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:53.123+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:53.576 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:53.574+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:54.113 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:54.111+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:54.566 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:54.563+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:55.131 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:55.128+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:55.594 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:55.591+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:56.177 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:56.174+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:56.556 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:56.554+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:57.147 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:57.146+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:57.585 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:57.583+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:58.189 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:58.187+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:58.626 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:58.624+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:59.218 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:48:59.216+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:48:59.577 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:48:59.575+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:00.261 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:00.260+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:00.536 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:00.535+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:01.237 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:01.236+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:01.487 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:01.486+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:02.260 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:02.258+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:02.452 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:02.451+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:03.306 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:03.305+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:03.473 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:03.472+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:04.312 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:04.311+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:04.471 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:04.470+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:05.290 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:05.289+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:05.491 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:05.490+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:06.302 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:06.301+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:06.450 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:06.449+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:07.312 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:07.311+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:07.448 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:07.447+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:08.308 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:08.307+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:08.434 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:08.434+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:09.338 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:09.337+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:09.385 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:09.384+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:10.302 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:10.301+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:10.373 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:10.372+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:11.312 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:11.311+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:11.363 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:11.362+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:12.303 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:12.302+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:12.318 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:12.317+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:13.257 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:13.256+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:13.310 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:13.309+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:14.244 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:14.243+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:14.349 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:14.348+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:15.280 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:15.279+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:15.371 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:15.369+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:16.248 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:16.247+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:16.412 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:16.411+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:17.246 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:17.245+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:17.454 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:17.453+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:18.271 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:18.270+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:18.456 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:18.455+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:19.264 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:19.263+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:19.506 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:19.505+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:20.315 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:20.313+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:20.544 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:20.542+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:21.280 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:21.278+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:21.518 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:21.516+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:22.246 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:22.244+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:22.530 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:22.528+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:23.231 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:23.229+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:23.510 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:23.508+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:24.232 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:24.230+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:24.550 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:24.548+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:25.206 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:25.204+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:25.514 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:25.512+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:26.180 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:26.178+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:26.474 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:26.472+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:27.231 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:27.229+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:27.510 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:27.508+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:28.200 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:28.198+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:28.537 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:28.534+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:29.152 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:29.150+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:29.495 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:29.492+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:30.183 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:30.181+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:30.538 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:30.536+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:31.212 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:31.210+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:31.564 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:31.562+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:32.260 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:32.258+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:32.571 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:32.569+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:33.255 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:33.253+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:33.574 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:33.572+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:34.278 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:34.276+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:34.538 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:34.536+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:35.262 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:35.260+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:35.545 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:35.543+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:36.240 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:36.238+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:36.527 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:36.525+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:37.194 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:37.192+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:37.541 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:37.539+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:38.178 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:38.176+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:38.567 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:38.565+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:39.188 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:39.186+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:39.568 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:39.566+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:40.219 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:40.217+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:40.613 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:40.611+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:41.238 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:41.236+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:41.586 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:41.584+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:42.236 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:42.234+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:42.582 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:42.580+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:43.232 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:43.230+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:43.576 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:43.574+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:44.276 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:44.274+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:44.619 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:44.617+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:45.271 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:45.269+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:45.648 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:45.646+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:46.237 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:46.235+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:46.690 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:46.688+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:47.238 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:47.236+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:47.698 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:47.696+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:48.234 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:48.232+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:48.708 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:48.706+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:49.273 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:49.271+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:49.724 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:49.722+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:50.240 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:50.238+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:50.726 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:50.724+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:51.193 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:51.191+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:51.773 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:51.771+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:52.230 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:52.228+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:52.732 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:52.730+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:53.204 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:53.202+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:53.700 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:53.697+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:54.162 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:54.160+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:54.708 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:54.706+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:55.210 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:55.208+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:55.666 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:55.664+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:56.187 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:56.185+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:56.637 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:56.635+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:57.178 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:57.176+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:57.631 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:57.629+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:58.206 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:58.204+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:58.646 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:58.645+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:59.222 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:49:59.220+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:49:59.607 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:49:59.606+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:00.214 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:00.213+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:00.640 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:00.639+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:01.184 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:01.182+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:01.596 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:01.595+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:02.148 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:02.147+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:02.617 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:02.616+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:03.170 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:03.168+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:03.572 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:03.571+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:04.195 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:04.194+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:04.559 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:04.558+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:05.174 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:05.173+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:05.537 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:05.536+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:06.208 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:06.207+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:06.516 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:06.515+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:07.212 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:07.211+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:07.564 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:07.563+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:08.208 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:08.207+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:08.553 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:08.552+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:09.210 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:09.209+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:09.539 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:09.538+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:10.206 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:10.205+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:10.515 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:10.514+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:11.233 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:11.232+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:11.546 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:11.545+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:12.278 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:12.277+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:12.577 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:12.576+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:13.316 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:13.315+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:13.572 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:13.571+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:14.271 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:14.270+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:14.551 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:14.550+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:15.320 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:15.319+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:15.530 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:15.529+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:16.284 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:16.283+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:16.508 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:16.507+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:17.327 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:17.326+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:17.548 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:17.547+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:18.325 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:18.324+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:18.551 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:18.550+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:19.373 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:19.371+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:19.601 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:19.599+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:20.384 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:20.382+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:20.634 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:20.632+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:21.336 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:21.334+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:21.675 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:21.673+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:22.306 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:22.304+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:22.704 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:22.701+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:23.292 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:23.290+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:23.664 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:23.662+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:24.335 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:24.333+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:24.714 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:24.712+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:25.372 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:25.370+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:25.714 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:25.712+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:26.378 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:26.376+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:26.726 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:26.724+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:27.387 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:27.385+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:27.728 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:27.726+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:28.344 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:28.342+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:28.738 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:28.736+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:29.296 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:29.294+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:29.730 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:29.728+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:30.341 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:30.339+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:30.750 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:30.748+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:31.302 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:31.300+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:31.737 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:31.735+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:32.272 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:32.270+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:32.700 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:32.698+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:33.284 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:33.282+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:33.692 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:33.690+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:34.316 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:34.314+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:34.650 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:34.647+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:35.290 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:35.288+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:35.615 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:35.613+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:36.269 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:36.267+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:36.623 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:36.621+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:37.271 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:37.269+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:37.582 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:37.580+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:38.314 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:38.312+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:38.628 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:38.626+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:39.318 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:39.316+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:39.656 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:39.654+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:40.357 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:40.355+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:40.705 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:40.703+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:41.374 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:41.372+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:41.676 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:41.674+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:42.410 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:42.408+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:42.658 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:42.656+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:43.366 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:43.364+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:43.626 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:43.623+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:44.316 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:44.314+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:44.637 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:44.635+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:45.332 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:45.330+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:45.644 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:45.642+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:46.303 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:46.301+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:46.623 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:46.621+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:47.259 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:47.257+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:47.592 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:47.589+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:48.232 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:48.230+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:48.559 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:48.557+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:49.240 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:49.238+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:49.521 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:49.519+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:50.250 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:50.248+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:50.489 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:50.486+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:51.217 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:51.215+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:51.495 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:51.493+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:52.175 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:52.173+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:52.492 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:52.489+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:53.163 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:53.161+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:53.497 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:53.495+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:54.116 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:54.114+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:54.530 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:54.527+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:55.073 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:55.071+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:55.576 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:55.575+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:56.026 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:56.024+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:56.625 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:56.623+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:56.989 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:56.988+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:57.654 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:57.653+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:57.963 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:57.962+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:58.681 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:58.680+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:58.925 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:58.924+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:59.667 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:50:59.666+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:50:59.940 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:50:59.939+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:00.651 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:00.650+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:00.943 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:00.941+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:01.683 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:01.682+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:01.934 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:01.933+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:02.642 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:02.641+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:02.959 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:02.958+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:03.595 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:03.594+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:03.922 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:03.921+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:04.589 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:04.586+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:04.890 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:04.889+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:05.637 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:05.636+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:05.873 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:05.872+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:06.594 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:06.593+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:06.899 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:06.898+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:07.614 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:07.612+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:07.906 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:07.905+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:08.640 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:08.639+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:08.910 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:08.909+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:09.639 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:09.638+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:09.933 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:09.932+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:10.625 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:10.624+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:10.928 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:10.927+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:11.672 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:11.671+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:11.930 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:11.929+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:12.663 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:12.662+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:12.910 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:12.909+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:13.667 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:13.666+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:13.899 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:13.898+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:14.656 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:14.655+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:14.935 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:14.934+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:15.628 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:15.627+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:15.949 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:15.948+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:16.672 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:16.671+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:16.993 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:16.992+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:17.713 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:17.711+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:18.017 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:18.015+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:18.728 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:18.726+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:19.033 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:19.031+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:19.767 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:19.765+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:20.017 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:20.014+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:20.804 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:20.802+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:20.972 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:20.970+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:21.763 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:21.761+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:21.997 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:21.995+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:22.785 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:22.783+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:22.977 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:22.975+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:23.835 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:23.833+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:24.025 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:24.023+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:24.839 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:24.837+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:24.999 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:24.997+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:25.821 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:25.819+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:25.966 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:25.964+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:26.815 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:26.813+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:26.942 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:26.940+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:27.778 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:27.776+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:27.907 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:27.904+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:28.799 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:28.797+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:28.888 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:28.886+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:29.771 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:29.769+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:29.882 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:29.880+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:30.751 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:30.748+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:30.881 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:30.879+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:31.794 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:31.792+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:31.929 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:31.926+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:32.759 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:32.757+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:32.889 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:32.887+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:33.719 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:33.717+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:33.889 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:33.887+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:34.690 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:34.688+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:34.927 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:34.925+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:35.713 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:35.711+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:35.963 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:35.961+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:36.750 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:36.748+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:36.975 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:36.973+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:37.791 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:37.789+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:37.954 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:37.952+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:38.759 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:38.756+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:38.942 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:38.940+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:39.762 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:39.760+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:39.934 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:39.932+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:40.775 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:40.773+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:40.973 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:40.971+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:41.738 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:41.736+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:41.964 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:41.962+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:42.740 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:42.738+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:42.996 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:42.994+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:43.739 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:43.737+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:43.954 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:43.952+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:44.765 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:44.763+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:45.004 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:45.002+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:45.753 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:45.751+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:45.996 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:45.994+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:46.755 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:46.753+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:46.967 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:46.965+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:47.774 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:47.772+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:48.000 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:47.998+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:48.729 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:48.727+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:48.965 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:48.963+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:49.737 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:49.735+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:50.007 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:50.005+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:50.744 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:50.742+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:50.982 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:50.979+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:51.757 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:51.755+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:52.002 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:52.000+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:52.796 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:52.794+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:53.032 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:53.030+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:53.787 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:53.784+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:54.007 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:54.005+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:54.768 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:54.766+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:55.031 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:55.029+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:55.772 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:55.771+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:56.013 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:56.012+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:56.796 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:56.795+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:57.044 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:57.043+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:57.808 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:57.806+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:58.010 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:58.009+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:58.840 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:58.839+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:59.059 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:51:59.058+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:51:59.851 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:51:59.850+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:00.098 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:00.097+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:00.860 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:00.859+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:01.066 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:01.065+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:01.840 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:01.839+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:02.028 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:02.027+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:02.858 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:02.856+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:02.986 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:02.985+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:03.809 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:03.808+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:04.018 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:04.017+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:04.838 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:04.837+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:05.002 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:05.000+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:05.848 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:05.847+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:06.049 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:06.046+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:06.804 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:06.803+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:07.036 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:07.035+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:07.809 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:07.808+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:07.998 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:07.997+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:08.839 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:08.838+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:09.005 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:09.004+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:09.811 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:09.810+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:10.034 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:10.033+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:10.837 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:10.836+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:11.010 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:11.009+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:11.797 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:11.796+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:12.003 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:12.002+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:12.837 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:12.836+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:13.008 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:13.007+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:13.822 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:13.821+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:13.983 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:13.982+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:14.824 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:14.823+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:15.001 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:15.000+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:15.862 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:15.860+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:16.022 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:16.020+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:16.895 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:16.893+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:17.012 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:17.010+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:17.870 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:17.868+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:17.976 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:17.974+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:18.860 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:18.856+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:18.956 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:18.954+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:19.902 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:19.900+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:20.003 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:20.000+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:20.878 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:20.876+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:21.010 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:21.008+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:21.855 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:21.853+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:21.970 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:21.968+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:22.876 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:22.874+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:22.981 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:22.980+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:23.916 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:23.914+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:23.974 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:23.972+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:24.903 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:24.901+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:24.974 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:24.972+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:25.931 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:25.929+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:25.969 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:25.967+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:26.904 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:26.902+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:26.973 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:26.971+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:27.886 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:27.884+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:27.952 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:27.950+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:28.904 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:28.902+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:28.905 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:28.903+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:29.922 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:29.920+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:29.952 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:29.950+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:30.895 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:30.893+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:30.951 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:30.949+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:31.882 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:31.880+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:31.986 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:31.984+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:32.878 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:32.876+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:32.961 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:32.959+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:33.922 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:33.920+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:33.944 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:33.942+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:34.948 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:34.946+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:34.949 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:34.947+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:35.958 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:35.956+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:35.988 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:35.986+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:36.966 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:36.964+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:36.989 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:36.987+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:37.986 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:37.984+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:38.033 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:38.031+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:39.004 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:39.002+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:39.016 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:39.014+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:40.011 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:40.009+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:40.015 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:40.013+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:41.015 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:41.013+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:41.053 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:41.051+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:42.013 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:42.011+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:42.098 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:42.095+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:43.062 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:43.060+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:43.123 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:43.122+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:44.112 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:44.110+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:44.169 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:44.167+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:45.066 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:45.064+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:45.120 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:45.118+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:46.024 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:46.022+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:46.132 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:46.130+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:47.025 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:47.023+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:47.122 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:47.120+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:48.048 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:48.046+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:48.128 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:48.126+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:49.018 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:49.016+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:49.121 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:49.119+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:50.006 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:50.003+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:50.135 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:50.133+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:51.027 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:51.025+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:51.181 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:51.179+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:52.075 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:52.072+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:52.164 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:52.163+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:53.058 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:53.056+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:53.138 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:53.137+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:54.075 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:54.073+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:54.133 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:54.133+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:55.031 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:55.030+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:55.158 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:55.157+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:56.012 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:56.011+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:56.173 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:56.172+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:57.010 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:57.009+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:57.198 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:57.197+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:58.037 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:58.036+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:58.214 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:58.213+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:59.084 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:52:59.083+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:52:59.224 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:52:59.223+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:00.040 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:00.039+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:00.180 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:00.179+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:01.046 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:01.045+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:01.191 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:01.190+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:02.046 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:02.045+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:02.185 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:02.184+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:03.015 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:03.014+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:03.228 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:03.227+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:04.041 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:04.039+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:04.214 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:04.213+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:05.014 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:05.013+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:05.195 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:05.194+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:06.005 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:06.004+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:06.184 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:06.183+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:07.017 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:07.016+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:07.138 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:07.137+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:07.980 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:07.979+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:08.099 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:08.098+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:08.933 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:08.931+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:09.080 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:09.079+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:09.911 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:09.910+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:10.041 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:10.040+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:10.948 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:10.947+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:11.038 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:11.037+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:11.944 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:11.943+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:12.013 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:12.012+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:12.968 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:12.967+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:13.046 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:13.045+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:13.920 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:13.918+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:14.034 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:14.032+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:14.952 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:14.950+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:15.063 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:15.061+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:15.952 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:15.950+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:16.070 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:16.068+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:16.937 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:16.935+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:17.100 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:17.098+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:17.906 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:17.904+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:18.095 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:18.092+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:18.926 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:18.924+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:19.145 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:19.143+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:19.959 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:19.957+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:20.158 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:20.156+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:20.981 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:20.978+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:21.198 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:21.196+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:21.952 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:21.950+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:22.166 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:22.164+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:22.979 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:22.977+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:23.142 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:23.140+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:23.934 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:23.932+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:24.154 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:24.152+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:24.956 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:24.954+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:25.154 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:25.152+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:26.001 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:25.999+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:26.188 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:26.186+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:27.018 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:27.016+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:27.226 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:27.224+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:27.980 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:27.978+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:28.243 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:28.241+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:28.996 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:28.994+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:29.200 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:29.198+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:30.000 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:29.998+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:30.249 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:30.247+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:31.004 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:31.002+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:31.294 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:31.292+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:31.978 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:31.976+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:32.333 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:32.331+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:33.013 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:33.011+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:33.328 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:33.326+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:34.038 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:34.036+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:34.377 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:34.375+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:35.054 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:35.052+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:35.373 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:35.371+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:36.042 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:36.040+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:36.399 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:36.397+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:37.001 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:36.999+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:37.390 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:37.388+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:37.965 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:37.963+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:38.392 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:38.390+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:38.937 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:38.935+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:39.399 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:39.397+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:39.964 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:39.962+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:40.415 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:40.413+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:41.002 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:41.000+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:41.455 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:41.453+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:42.038 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:42.036+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:42.422 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:42.420+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:43.082 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:43.080+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:43.460 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:43.458+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:44.050 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:44.048+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:44.445 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:44.443+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:45.041 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:45.039+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:45.470 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:45.468+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:46.034 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:46.032+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:46.432 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:46.430+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:47.065 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:47.063+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:47.434 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:47.432+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:48.041 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:48.039+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:48.436 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:48.434+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:49.032 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:49.030+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:49.441 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:49.439+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:50.019 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:50.017+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:50.466 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:50.464+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:51.067 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:51.065+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:51.470 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:51.468+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:52.114 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:52.113+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:52.442 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:52.441+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:53.073 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:53.072+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:53.459 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:53.458+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:54.057 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:54.056+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:54.476 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:54.475+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:55.095 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:55.094+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:55.440 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:55.438+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:56.105 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:56.104+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:56.428 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:56.427+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:57.109 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:57.108+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:57.475 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:57.474+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:58.091 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:58.090+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:58.457 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:58.456+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:59.102 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:53:59.101+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:53:59.478 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:53:59.476+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:00.131 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:00.130+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:00.525 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:00.524+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:01.158 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:01.157+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:01.529 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:01.528+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:02.114 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:02.113+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:02.499 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:02.498+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:03.084 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:03.083+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:03.537 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:03.536+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:04.071 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:04.070+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:04.540 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:04.539+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:05.055 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:05.054+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:05.525 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:05.524+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:06.084 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:06.082+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:06.545 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:06.544+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:07.131 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:07.130+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:07.556 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:07.555+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:08.113 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:08.111+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:08.521 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:08.520+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:09.113 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:09.112+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:09.549 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:09.548+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:10.128 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:10.127+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:10.574 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:10.573+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:11.132 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:11.131+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:11.613 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:11.612+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:12.159 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:12.157+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:12.643 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:12.641+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:13.144 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:13.141+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:13.655 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:13.653+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:14.115 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:14.113+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:14.643 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:14.641+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:15.067 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:15.065+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:15.674 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:15.672+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:16.033 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:16.031+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:16.709 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:16.707+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:17.039 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:17.037+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:17.661 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:17.659+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:18.013 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:18.011+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:18.703 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:18.701+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:18.981 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:18.979+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:19.735 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:19.733+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:20.004 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:20.002+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:20.745 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:20.743+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:20.963 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:20.961+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:21.726 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:21.724+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:21.967 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:21.965+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:22.769 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:22.767+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:23.003 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:23.001+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:23.811 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:23.809+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:23.987 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:23.984+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:24.836 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:24.834+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:24.961 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:24.959+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:25.879 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:25.877+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:25.974 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:25.972+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:26.843 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:26.840+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:26.938 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:26.936+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:27.816 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:27.814+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:27.985 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:27.983+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:28.809 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:28.807+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:29.028 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:29.026+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:29.770 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:29.768+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:30.076 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:30.074+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:30.743 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:30.741+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:31.093 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:31.091+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:31.748 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:31.746+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:32.051 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:32.049+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:32.765 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:32.763+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:33.015 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:33.013+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:33.731 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:33.729+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:34.043 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:34.041+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:34.694 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:34.692+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:35.004 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:35.002+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:35.711 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:35.709+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:36.031 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:36.029+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:36.760 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:36.759+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:37.059 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:37.057+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:37.773 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:37.771+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:38.080 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:38.077+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:38.799 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:38.797+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:39.073 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:39.071+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:39.780 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:39.778+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:40.106 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:40.104+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:40.740 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:40.738+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:41.107 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:41.105+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:41.745 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:41.743+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:42.091 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:42.089+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:42.787 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:42.785+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:43.097 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:43.095+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:43.758 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:43.756+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:44.081 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:44.078+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:44.727 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:44.725+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:45.042 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:45.040+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:45.766 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:45.764+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:46.073 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:46.071+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:46.774 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:46.772+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:47.024 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:47.022+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:47.749 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:47.747+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:47.989 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:47.987+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:48.731 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:48.729+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:49.017 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:49.015+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:49.766 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:49.765+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:49.981 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:49.980+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:50.794 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:50.793+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:50.973 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:50.972+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:51.776 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:51.775+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:51.978 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:51.977+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:52.764 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:52.763+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:52.957 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:52.956+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:53.731 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:53.730+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:53.930 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:53.929+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:54.696 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:54.695+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:54.930 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:54.929+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:55.673 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:55.672+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:55.889 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:55.888+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:56.644 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:56.643+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:56.918 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:56.917+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:57.615 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:57.614+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:57.915 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:57.914+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:58.640 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:58.639+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:58.884 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:58.883+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:59.614 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:54:59.613+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:54:59.887 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:54:59.886+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:00.610 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:00.609+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:00.879 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:00.878+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:01.614 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:01.613+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:01.852 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:01.850+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:02.602 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:02.600+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:02.878 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:02.877+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:03.593 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:03.592+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:03.928 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:03.918+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:04.638 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:04.637+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:04.892 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:04.891+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:05.656 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:05.654+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:05.904 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:05.903+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:06.668 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:06.667+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:06.900 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:06.899+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:07.692 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:07.691+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:07.860 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:07.859+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:08.679 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:08.678+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:08.898 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:08.897+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:09.662 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:09.661+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:09.876 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:09.875+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:10.709 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:10.707+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:10.868 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:10.866+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:11.752 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:11.750+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:11.895 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:11.893+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:12.732 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:12.730+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:12.924 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:12.922+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:13.756 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:13.754+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:13.920 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:13.918+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:14.716 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:14.714+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:14.933 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:14.931+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:15.722 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:15.720+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:15.935 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:15.933+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:16.736 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:16.734+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:16.956 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:16.954+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:17.735 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:17.733+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:17.988 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:17.986+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:18.784 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:18.781+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:19.002 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:19.000+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:19.785 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:19.783+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:19.990 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:19.988+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:20.803 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:20.801+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:20.942 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:20.940+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:21.794 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:21.792+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:21.986 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:21.984+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:22.837 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:22.834+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:22.964 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:22.962+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:23.834 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:23.832+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:23.985 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:23.983+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:24.839 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:24.836+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:24.978 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:24.976+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:25.830 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:25.828+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:25.980 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:25.978+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:26.806 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:26.804+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:26.954 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:26.952+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:27.844 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:27.842+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:27.923 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:27.921+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:28.889 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:28.887+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:28.936 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:28.934+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:29.864 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:29.862+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:29.906 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:29.904+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:30.816 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:30.814+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:30.859 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:30.857+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:31.852 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:31.850+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:31.860 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:31.858+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:32.842 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:32.840+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:32.892 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:32.890+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:33.866 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:33.864+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:33.896 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:33.894+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:34.883 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:34.880+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:34.910 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:34.908+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:35.850 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:35.847+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:35.870 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:35.868+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:36.889 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:36.887+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:36.910 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:36.908+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:37.859 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:37.857+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:37.937 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:37.936+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:38.852 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:38.850+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:38.926 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:38.924+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:39.896 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:39.894+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:39.938 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:39.936+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:40.934 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:40.932+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:40.960 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:40.958+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:41.893 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:41.891+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:41.972 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:41.970+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:42.937 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:42.935+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:42.938 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:42.936+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:43.932 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:43.930+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:43.980 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:43.978+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:44.934 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:44.932+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:44.981 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:44.979+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:45.894 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:45.892+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:45.950 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:45.948+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:46.878 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:46.875+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:46.912 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:46.910+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:47.874 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:47.872+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:47.961 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:47.959+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:48.879 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:48.877+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:48.964 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:48.963+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:49.904 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:49.903+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:49.975 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:49.974+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:50.888 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:50.887+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:50.953 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:50.952+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:51.867 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:51.866+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:51.995 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:51.994+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:52.843 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:52.842+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:52.973 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:52.972+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:53.844 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:53.843+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:53.929 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:53.928+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:54.819 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:54.818+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:54.975 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:54.974+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:55.846 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:55.845+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:55.953 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:55.952+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:56.801 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:56.797+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:56.994 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:56.993+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:57.751 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:57.750+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:57.968 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:57.967+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:58.719 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:58.718+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:58.951 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:58.950+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:59.688 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:55:59.687+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:55:59.936 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:55:59.935+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:00.723 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:00.722+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:00.973 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:00.972+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:01.705 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:01.704+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:02.005 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:02.004+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:02.755 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:02.754+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:03.017 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:03.015+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:03.725 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:03.724+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:04.018 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:04.017+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:04.739 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:04.738+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:05.003 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:05.002+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:05.785 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:05.784+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:05.964 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:05.963+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:06.763 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:06.762+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:06.977 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:06.976+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:07.782 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:07.781+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:08.013 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:08.012+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:08.787 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:08.785+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:09.026 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:09.024+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:09.765 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:09.762+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:10.023 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:10.021+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:10.738 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:10.736+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:10.980 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:10.978+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:11.735 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:11.733+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:11.951 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:11.948+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:12.745 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:12.743+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:12.960 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:12.958+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:13.697 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:13.695+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:13.969 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:13.967+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:14.664 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:14.661+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:14.956 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:14.954+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:15.640 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:15.638+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:15.987 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:15.985+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:16.639 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:16.637+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:16.968 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:16.966+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:17.598 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:17.596+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:17.991 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:17.989+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:18.614 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:18.611+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:18.948 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:18.946+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:19.623 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:19.621+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:19.940 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:19.938+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:20.598 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:20.596+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:20.907 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:20.905+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:21.574 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:21.571+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:21.877 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:21.875+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:22.607 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:22.605+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:22.911 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:22.909+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:23.582 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:23.580+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:23.915 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:23.913+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:24.588 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:24.586+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:24.880 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:24.878+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:25.577 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:25.575+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:25.859 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:25.857+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:26.571 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:26.569+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:26.854 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:26.852+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:27.561 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:27.559+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:27.822 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:27.820+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:28.529 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:28.527+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:28.816 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:28.813+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:29.493 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:29.491+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:29.816 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:29.814+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:30.526 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:30.524+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:30.834 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:30.832+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:31.550 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:31.548+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:31.840 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:31.838+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:32.577 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:32.575+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:32.794 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:32.792+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:33.548 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:33.546+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:33.783 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:33.781+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:34.593 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:34.591+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:34.809 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:34.807+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:35.617 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:35.615+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:35.820 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:35.818+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:36.615 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:36.613+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:36.826 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:36.824+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:37.605 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:37.603+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:37.830 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:37.828+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:38.637 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:38.635+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:38.822 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:38.820+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:39.603 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:39.601+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:39.866 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:39.864+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:40.578 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:40.577+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:40.866 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:40.864+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:41.609 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:41.607+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:41.877 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:41.875+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:42.656 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:42.654+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:42.854 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:42.852+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:43.657 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:43.655+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:43.899 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:43.897+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:44.704 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:44.702+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:44.857 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:44.855+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:45.700 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:45.698+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:45.871 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:45.869+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:46.729 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:46.728+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:46.909 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:46.907+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:47.720 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:47.719+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:47.883 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:47.881+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:48.693 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:48.692+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:48.877 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:48.876+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:49.714 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:49.713+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:49.885 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:49.884+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:50.715 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:50.714+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:50.897 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:50.896+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:51.671 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:51.670+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:51.867 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:51.866+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:52.651 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:52.650+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:52.876 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:52.874+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:53.672 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:53.671+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:53.850 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:53.849+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:54.666 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:54.665+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:54.802 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:54.801+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:55.623 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:55.622+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:55.816 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:55.815+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:56.606 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:56.605+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:56.831 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:56.830+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:57.649 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:57.648+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:57.816 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:57.815+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:58.684 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:58.683+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:58.787 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:58.786+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:59.657 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:56:59.655+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:56:59.784 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:56:59.783+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:00.674 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:00.672+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:00.738 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:00.737+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:01.635 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:01.634+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:01.693 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:01.692+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:02.633 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:02.632+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:02.649 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:02.648+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:03.609 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:03.608+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:03.685 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:03.684+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:04.645 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:04.644+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:04.722 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:04.721+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:05.664 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:05.662+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:05.720 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:05.719+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:06.663 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:06.661+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:06.688 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:06.686+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:07.654 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:07.652+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:07.699 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:07.697+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:08.631 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:08.629+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:08.743 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:08.741+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:09.627 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:09.625+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:09.776 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:09.774+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:10.678 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:10.676+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:10.821 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:10.819+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:11.701 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:11.699+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:11.789 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:11.787+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:12.685 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:12.683+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:12.760 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:12.758+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:13.640 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:13.638+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:13.797 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:13.795+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:14.599 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:14.596+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:14.756 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:14.753+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:15.564 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:15.562+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:15.773 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:15.771+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:16.542 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:16.540+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:16.734 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:16.732+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:17.533 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:17.531+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:17.689 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:17.687+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:18.530 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:18.528+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:18.732 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:18.730+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:19.542 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:19.540+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:19.688 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:19.686+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:20.573 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:20.570+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:20.642 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:20.641+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:21.531 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:21.529+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:21.647 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:21.644+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:22.556 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:22.554+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:22.687 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:22.685+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:23.589 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:23.587+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:23.709 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:23.707+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:24.589 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:24.587+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:24.697 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:24.695+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:25.544 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:25.541+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:25.725 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:25.723+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:26.565 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:26.563+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:26.720 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:26.718+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:27.610 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:27.608+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:27.673 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:27.671+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:28.639 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:28.637+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:28.663 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:28.661+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:29.621 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:29.619+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:29.627 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:29.625+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:30.622 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:30.619+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:30.630 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:30.628+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:31.587 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:31.585+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:31.615 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:31.613+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:32.546 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:32.544+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:32.631 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:32.629+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:33.541 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:33.539+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:33.637 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:33.635+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:34.525 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:34.522+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:34.687 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:34.685+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:35.547 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:35.545+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:35.682 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:35.680+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:36.505 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:36.503+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:36.706 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:36.702+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:37.532 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:37.529+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:37.717 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:37.715+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:38.579 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:38.577+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:38.730 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:38.728+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:39.615 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:39.613+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:39.734 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:39.732+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:40.657 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:40.655+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:40.757 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:40.755+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:41.672 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:41.670+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:41.807 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:41.805+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:42.660 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:42.658+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:42.832 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:42.830+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:43.621 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:43.620+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:43.825 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:43.823+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:44.653 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:44.652+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:44.823 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:44.822+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:45.640 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:45.639+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:45.841 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:45.840+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:46.593 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:46.592+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:46.818 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:46.817+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:47.596 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:47.595+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:47.818 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:47.817+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:48.625 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:48.624+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:48.799 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:48.798+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:49.636 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:49.635+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:49.772 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:49.771+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:50.588 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:50.587+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:50.790 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:50.789+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:51.544 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:51.543+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:51.809 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:51.808+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:52.547 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:52.546+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:52.809 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:52.807+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:53.544 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:53.542+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:53.813 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:53.812+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:54.506 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:54.505+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:54.839 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:54.838+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:55.539 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:55.538+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:55.847 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:55.846+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:56.533 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:56.532+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:56.805 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:56.804+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:57.508 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:57.507+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:57.786 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:57.785+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:58.515 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:58.514+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:58.817 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:58.816+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:59.508 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:57:59.507+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:57:59.790 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:57:59.789+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:00.544 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:00.543+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:00.818 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:00.817+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:01.550 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:01.549+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:01.777 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:01.776+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:02.532 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:02.531+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:02.750 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:02.749+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:03.557 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:03.556+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:03.724 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:03.723+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:04.529 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:04.527+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:04.766 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:04.764+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:05.576 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:05.574+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:05.816 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:05.814+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:06.541 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:06.539+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:06.805 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:06.803+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:07.530 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:07.528+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:07.850 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:07.848+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:08.486 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:08.484+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:08.812 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:08.810+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:09.481 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:09.479+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:09.826 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:09.824+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:10.482 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:10.480+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:10.798 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:10.797+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:11.448 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:11.446+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:11.754 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:11.752+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:12.455 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:12.453+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:12.767 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:12.765+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:13.438 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:13.436+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:13.755 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:13.754+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:14.410 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:14.408+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:14.766 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:14.762+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:15.448 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:15.446+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:15.726 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:15.723+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:16.433 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:16.431+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:16.683 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:16.681+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:17.445 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:17.443+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:17.660 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:17.657+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:18.408 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:18.406+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:18.674 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:18.672+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:19.429 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:19.427+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:19.642 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:19.640+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:20.469 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:20.467+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:20.608 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:20.606+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:21.506 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:21.504+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:21.613 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:21.611+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:22.502 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:22.500+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:22.637 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:22.635+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:23.516 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:23.514+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:23.687 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:23.685+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:24.502 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:24.500+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:24.702 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:24.699+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:25.512 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:25.510+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:25.717 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:25.715+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:26.522 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:26.520+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:26.760 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:26.758+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:27.542 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:27.540+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:27.774 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:27.772+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:28.566 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:28.564+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:28.804 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:28.802+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:29.574 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:29.571+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:29.818 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:29.816+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:30.602 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:30.600+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:30.816 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:30.814+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:31.569 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:31.567+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:31.785 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:31.783+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:32.522 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:32.520+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:32.833 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:32.831+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:33.513 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:33.511+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:33.789 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:33.784+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:34.510 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:34.508+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:34.746 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:34.744+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:35.489 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:35.487+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:35.746 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:35.744+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:36.504 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:36.502+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:36.778 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:36.776+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:37.548 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:37.546+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:37.796 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:37.794+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:38.562 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:38.560+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:38.759 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:38.757+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:39.525 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:39.523+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:39.717 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:39.715+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:40.504 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:40.502+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:40.758 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:40.756+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:41.482 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:41.480+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:41.750 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:41.748+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:42.462 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:42.460+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:42.718 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:42.717+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:43.490 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:43.489+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:43.714 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:43.713+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:44.521 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:44.520+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:44.680 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:44.679+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:45.526 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:45.525+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:45.698 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:45.697+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:46.534 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:46.533+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:46.741 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:46.740+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:47.501 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:47.500+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:47.710 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:47.709+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:48.541 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:48.540+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:48.685 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:48.685+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:49.546 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:49.545+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:49.719 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:49.718+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:50.539 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:50.538+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:50.700 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:50.699+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:51.556 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:51.555+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:51.749 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:51.748+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:52.516 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:52.515+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:52.749 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:52.748+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:53.516 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:53.515+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:53.761 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:53.760+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:54.484 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:54.483+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:54.752 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:54.751+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:55.458 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:55.456+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:55.727 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:55.726+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:56.455 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:56.454+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:56.739 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:56.737+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:57.446 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:57.445+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:57.751 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:57.750+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:58.493 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:58.492+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:58.728 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:58.726+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:59.480 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:58:59.479+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:58:59.770 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:58:59.769+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:00.497 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:00.496+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:00.797 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:00.796+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:01.540 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:01.539+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:01.752 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:01.751+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:02.538 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:02.536+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:02.778 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:02.776+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:03.512 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:03.510+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:03.778 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:03.776+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:04.542 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:04.540+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:04.738 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:04.736+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:05.591 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:05.589+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:05.759 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:05.757+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:06.545 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:06.543+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:06.786 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:06.784+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:07.505 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:07.503+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:07.745 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:07.743+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:08.546 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:08.544+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:08.721 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:08.720+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:09.549 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:09.547+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:09.757 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:09.755+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:10.532 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:10.530+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:10.742 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:10.740+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:11.567 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:11.565+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:11.768 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:11.766+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:12.554 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:12.552+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:12.780 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:12.777+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:13.508 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:13.506+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:13.794 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:13.791+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:14.484 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:14.482+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:14.826 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:14.823+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:15.505 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:15.503+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:15.846 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:15.844+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:16.544 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:16.542+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:16.824 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:16.822+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:17.528 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:17.526+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:17.805 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:17.803+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:18.546 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:18.544+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:18.778 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:18.776+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:19.579 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:19.577+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:19.785 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:19.783+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:20.626 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:20.624+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:20.827 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:20.825+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:21.612 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:21.610+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:21.876 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:21.874+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:22.654 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:22.652+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:22.870 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:22.868+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:23.680 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:23.678+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:23.836 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:23.834+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:24.702 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:24.700+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:24.822 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:24.820+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:25.727 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:25.725+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:25.825 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:25.823+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:26.752 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:26.750+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:26.832 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:26.830+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:27.710 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:27.707+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:27.798 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:27.796+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:28.680 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:28.678+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:28.833 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:28.830+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:29.688 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:29.686+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:29.808 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:29.806+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:30.722 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:30.720+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:30.831 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:30.829+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:31.744 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:31.742+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:31.827 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:31.825+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:32.763 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:32.761+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:32.860 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:32.858+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:33.717 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:33.715+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:33.842 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:33.840+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:34.741 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:34.739+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:34.867 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:34.865+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:35.740 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:35.738+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:35.829 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:35.827+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:36.788 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:36.786+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:36.877 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:36.875+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:37.792 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:37.790+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:37.917 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:37.915+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:38.762 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:38.760+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:38.904 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:38.902+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:39.793 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:39.790+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:39.955 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:39.953+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:40.774 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:40.773+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:40.964 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:40.963+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:41.748 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:41.746+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:41.932 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:41.931+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:42.745 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:42.744+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:42.950 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:42.949+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:43.777 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:43.776+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:43.989 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:43.988+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:44.751 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:44.749+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:45.025 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:45.024+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:45.773 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:45.772+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:45.999 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:45.998+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:46.810 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:46.808+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:46.964 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:46.963+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:47.851 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:47.850+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:47.947 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:47.946+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:48.883 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:48.882+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:48.926 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:48.925+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:49.861 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:49.859+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:49.929 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:49.928+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:50.815 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:50.814+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:50.917 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:50.916+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:51.789 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:51.788+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:51.929 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:51.928+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:52.760 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:52.759+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:52.977 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:52.976+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:53.726 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:53.726+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:53.963 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:53.962+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:54.681 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:54.680+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:54.997 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:54.996+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:55.713 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:55.712+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:56.009 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:56.008+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:56.672 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:56.671+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:56.987 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:56.986+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:57.685 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:57.684+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:57.998 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:57.997+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:58.677 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:58.676+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:58.989 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:58.988+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:59.678 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T21:59:59.677+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T21:59:59.979 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T21:59:59.978+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:00.631 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:00.629+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:01.020 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:01.018+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:01.603 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:01.601+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:02.056 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:02.054+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:02.631 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:02.628+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:03.051 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:03.049+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:03.649 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:03.647+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:04.061 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:04.059+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:04.603 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:04.601+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:05.032 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:05.030+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:05.585 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:05.583+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:06.057 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:06.055+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:06.541 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:06.539+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:07.101 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:07.099+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:07.492 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:07.490+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:08.112 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:08.110+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:08.447 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:08.444+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:09.063 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:09.061+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:09.411 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:09.409+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:10.015 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:10.013+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:10.442 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:10.440+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:10.974 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:10.972+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:11.487 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:11.485+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:11.942 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:11.940+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:12.523 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:12.521+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:12.984 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:12.982+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:13.520 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:13.518+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:13.974 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:13.972+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:14.505 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:14.503+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:14.945 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:14.943+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:15.472 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:15.469+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:15.944 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:15.942+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:16.427 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:16.425+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:16.961 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:16.959+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:17.389 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:17.387+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:17.937 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:17.935+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:18.437 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:18.434+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:18.985 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:18.983+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:19.443 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:19.441+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:19.963 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:19.961+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:20.452 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:20.450+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:20.924 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:20.922+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:21.417 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:21.415+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:21.937 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:21.935+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:22.417 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:22.415+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:22.974 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:22.972+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:23.452 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:23.450+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:23.927 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:23.925+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:24.418 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:24.416+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:24.975 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:24.973+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:25.426 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:25.424+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:26.007 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:26.005+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:26.452 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:26.450+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:27.041 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:27.039+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:27.463 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:27.461+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:28.038 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:28.036+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:28.456 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:28.454+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:29.029 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:29.027+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:29.439 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:29.437+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:30.026 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:30.024+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:30.457 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:30.455+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:31.007 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:31.005+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:31.411 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:31.409+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:31.961 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:31.959+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:32.457 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:32.453+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:32.957 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:32.955+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:33.432 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:33.430+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:34.003 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:34.001+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:34.424 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:34.423+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:35.023 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:35.021+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:35.396 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:35.394+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:36.063 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:36.061+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:36.414 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:36.413+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:37.040 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:37.038+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:37.428 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:37.427+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:38.078 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:38.076+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:38.436 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:38.435+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:39.094 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:39.093+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:39.483 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:39.483+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:40.124 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:40.123+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:40.468 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:40.467+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:41.097 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:41.096+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:41.492 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:41.491+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:42.067 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:42.066+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:42.470 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:42.468+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:43.114 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:43.113+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:43.481 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:43.480+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:44.133 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:44.132+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:44.456 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:44.455+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:45.169 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:45.168+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:45.472 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:45.471+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:46.188 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:46.187+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:46.441 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:46.440+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:47.201 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:47.200+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:47.430 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:47.429+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:48.226 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:48.225+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:48.468 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:48.467+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:49.179 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:49.178+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:49.423 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:49.421+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:50.138 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:50.137+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:50.382 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:50.381+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:51.148 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:51.147+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:51.412 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:51.411+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:52.184 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:52.183+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:52.438 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:52.437+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:53.168 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:53.167+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:53.453 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:53.452+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:54.202 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:54.201+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:54.449 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:54.448+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:55.212 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:55.211+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:55.433 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:55.431+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:56.237 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:56.236+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:56.433 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:56.432+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:57.282 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:57.281+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:57.438 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:57.437+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:58.256 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:58.255+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:58.447 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:58.445+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:59.249 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:00:59.247+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:00:59.468 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:00:59.466+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:00.221 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:00.219+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:00.424 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:00.422+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:01.204 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:01.202+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:01.423 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:01.421+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:02.179 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:02.177+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:02.456 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:02.454+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:03.140 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:03.138+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:03.480 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:03.478+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:04.153 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:04.151+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:04.434 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:04.432+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:05.173 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:05.171+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:05.471 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:05.469+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:06.171 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:06.169+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:06.469 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:06.467+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:07.197 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:07.195+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:07.494 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:07.492+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:08.201 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:08.199+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:08.474 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:08.472+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:09.244 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:09.242+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:09.448 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:09.446+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:10.205 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:10.203+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:10.419 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:10.417+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:11.210 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:11.208+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:11.448 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:11.446+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:12.180 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:12.178+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:12.466 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:12.464+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:13.215 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:13.213+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:13.470 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:13.468+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:14.185 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:14.183+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:14.422 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:14.420+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:15.164 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:15.162+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:15.391 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:15.389+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:16.126 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:16.124+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:16.387 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:16.385+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:17.104 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:17.102+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:17.420 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:17.418+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:18.064 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:18.062+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:18.452 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:18.450+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:19.076 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:19.074+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:19.451 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:19.449+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:20.076 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:20.074+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:20.494 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:20.491+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:21.080 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:21.078+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:21.522 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:21.520+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:22.067 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:22.064+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:22.474 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:22.472+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:23.064 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:23.062+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:23.518 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:23.513+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:24.076 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:24.073+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:24.496 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:24.494+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:25.074 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:25.072+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:25.457 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:25.454+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:26.033 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:26.031+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:26.432 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:26.430+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:27.081 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:27.078+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:27.404 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:27.402+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:28.038 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:28.035+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:28.360 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:28.358+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:29.084 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:29.082+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:29.413 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:29.404+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:30.119 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:30.117+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:30.359 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:30.357+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:31.168 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:31.166+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:31.317 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:31.315+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:32.134 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:32.132+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:32.365 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:32.363+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:33.130 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:33.128+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:33.352 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:33.350+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:34.175 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:34.173+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:34.356 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:34.354+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:35.219 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:35.217+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:35.386 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:35.384+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:36.191 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:36.189+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:36.367 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:36.365+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:37.234 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:37.233+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:37.332 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:37.331+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:38.213 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:38.212+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:38.335 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:38.334+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:39.263 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:39.262+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:39.291 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:39.290+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:40.240 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:40.239+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:40.331 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:40.330+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:41.265 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:41.264+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:41.354 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:41.354+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:42.306 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:42.305+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:42.402 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:42.401+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:43.297 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:43.296+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:43.357 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:43.356+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:44.318 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:44.316+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:44.334 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:44.334+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:45.273 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:45.271+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:45.351 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:45.350+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:46.317 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:46.316+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:46.401 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:46.400+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:47.336 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:47.335+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:47.366 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:47.365+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:48.333 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:48.332+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:48.405 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:48.404+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:49.369 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:49.368+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:49.373 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:49.373+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:50.339 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:50.338+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:50.351 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:50.350+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:51.317 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:51.316+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:51.387 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:51.386+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:52.307 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:52.306+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:52.425 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:52.424+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:53.278 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:53.277+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:53.456 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:53.455+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:54.245 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:54.244+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:54.496 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:54.495+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:55.237 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:55.236+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:55.534 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:55.533+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:56.198 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:56.198+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:56.506 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:56.505+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:57.246 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:57.244+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:57.529 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:57.527+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:58.292 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:58.289+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:58.540 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:58.538+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:59.305 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:01:59.303+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:01:59.545 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:01:59.543+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:00.342 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:00.340+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:00.556 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:00.554+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:01.380 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:01.378+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:01.524 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:01.522+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:02.359 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:02.357+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:02.499 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:02.497+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:03.363 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:03.361+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:03.546 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:03.544+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:04.396 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:04.394+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:04.555 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:04.553+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:05.364 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:05.362+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:05.537 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:05.534+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:06.383 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:06.380+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:06.521 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:06.519+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:07.360 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:07.358+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:07.543 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:07.541+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:08.346 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:08.344+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:08.537 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:08.535+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:09.334 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:09.332+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:09.511 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:09.509+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:10.373 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:10.371+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:10.553 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:10.550+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:11.403 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:11.401+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:11.579 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:11.577+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:12.449 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:12.447+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:12.577 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:12.575+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:13.452 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:13.450+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:13.576 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:13.574+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:14.449 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:14.447+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:14.543 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:14.541+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:15.483 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:15.481+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:15.537 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:15.535+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:16.532 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:16.529+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:16.549 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:16.547+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:17.507 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:17.505+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:17.576 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:17.574+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:18.515 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:18.513+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:18.569 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:18.567+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:19.513 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:19.511+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:19.523 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:19.521+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:20.474 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:20.472+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:20.527 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:20.525+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:21.425 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:21.423+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:21.529 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:21.527+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:22.406 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:22.404+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:22.541 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:22.539+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:23.434 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:23.432+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:23.572 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:23.570+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:24.407 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:24.405+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:24.541 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:24.539+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:25.364 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:25.362+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:25.523 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:25.521+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:26.316 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:26.314+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:26.530 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:26.528+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:27.347 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:27.345+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:27.527 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:27.525+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:28.376 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:28.373+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:28.533 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:28.531+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:29.387 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:29.385+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:29.567 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:29.565+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:30.376 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:30.374+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:30.520 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:30.518+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:31.355 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:31.353+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:31.567 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:31.565+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:32.323 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:32.321+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:32.561 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:32.559+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:33.337 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:33.335+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:33.574 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:33.572+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:34.318 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:34.316+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:34.538 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:34.536+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:35.307 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:35.306+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:35.550 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:35.549+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:36.332 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:36.331+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:36.589 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:36.588+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:37.339 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:37.338+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:37.590 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:37.588+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:38.325 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:38.323+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:38.603 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:38.602+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:39.337 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:39.336+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:39.597 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:39.596+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:40.322 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:40.321+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:40.627 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:40.626+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:41.342 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:41.341+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:41.618 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:41.616+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:42.346 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:42.345+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:42.642 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:42.640+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:43.311 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:43.310+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:43.692 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:43.691+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:44.330 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:44.329+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:44.700 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:44.699+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:45.373 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:45.372+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:45.722 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:45.720+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:46.392 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:46.392+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:46.673 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:46.672+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:47.369 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:47.368+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:47.671 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:47.670+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:48.416 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:48.415+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:48.681 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:48.680+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:49.387 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:49.386+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:49.662 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:49.661+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:50.429 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:50.428+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:50.651 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:50.650+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:51.449 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:51.448+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:51.632 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:51.631+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:52.448 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:52.447+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:52.632 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:52.631+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:53.405 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:53.404+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:53.663 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:53.662+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:54.406 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:54.405+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:54.650 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:54.649+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:55.405 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:55.404+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:55.671 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:55.669+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:56.366 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:56.364+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:56.679 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:56.677+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:57.361 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:57.358+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:57.683 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:57.681+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:58.343 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:58.341+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:58.673 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:58.671+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:59.325 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:02:59.323+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:02:59.646 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:02:59.644+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:00.287 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:00.285+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:00.689 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:00.687+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:01.240 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:01.238+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:01.695 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:01.693+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:02.217 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:02.214+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:02.678 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:02.676+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:03.219 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:03.217+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:03.633 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:03.631+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:04.172 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:04.170+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:04.621 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:04.619+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:05.145 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:05.143+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:05.635 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:05.633+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:06.142 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:06.141+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:06.609 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:06.607+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:07.185 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:07.183+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:07.642 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:07.640+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:08.212 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:08.210+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:08.684 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:08.682+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:09.176 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:09.174+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:09.727 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:09.725+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:10.207 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:10.205+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:10.747 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:10.745+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:11.185 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:11.183+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:11.753 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:11.751+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:12.179 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:12.177+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:12.750 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:12.748+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:13.185 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:13.183+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:13.753 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:13.751+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:14.190 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:14.188+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:14.803 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:14.801+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:15.146 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:15.144+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:15.781 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:15.779+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:16.168 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:16.166+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:16.759 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:16.757+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:17.146 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:17.144+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:17.736 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:17.734+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:18.149 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:18.147+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:18.726 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:18.724+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:19.129 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:19.127+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:19.746 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:19.745+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:20.138 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:20.136+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:20.792 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:20.790+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:21.177 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:21.175+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:21.836 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:21.834+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:22.221 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:22.219+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:22.814 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:22.812+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:23.207 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:23.205+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:23.784 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:23.782+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:24.216 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:24.214+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:24.749 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:24.747+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:25.237 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:25.235+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:25.792 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:25.790+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:26.235 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:26.233+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:26.831 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:26.829+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:27.196 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:27.194+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:27.845 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:27.843+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:28.216 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:28.214+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:28.803 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:28.801+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:29.235 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:29.233+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:29.825 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:29.823+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:30.244 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:30.243+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:30.800 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:30.798+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:31.201 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:31.199+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:31.781 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:31.779+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:32.181 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:32.179+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:32.791 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:32.790+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:33.222 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:33.220+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:33.839 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:33.838+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:34.192 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:34.191+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:34.867 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:34.866+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:35.191 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:35.190+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:35.843 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:35.842+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:36.187 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:36.186+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:36.891 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:36.890+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:37.199 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:37.198+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:37.898 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:37.897+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:38.230 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:38.229+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:38.916 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:38.915+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:39.184 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:39.183+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:39.894 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:39.893+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:40.148 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:40.147+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:40.934 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:40.933+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:41.193 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:41.192+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:41.951 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:41.950+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:42.153 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:42.152+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:42.940 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:42.939+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:43.156 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:43.155+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:43.904 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:43.903+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:44.150 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:44.149+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:44.871 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:44.870+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:45.179 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:45.178+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:45.861 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:45.860+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:46.137 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:46.134+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:46.852 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:46.851+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:47.165 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:47.165+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:47.881 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:47.880+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:48.117 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:48.116+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:48.838 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:48.837+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:49.121 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:49.120+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:49.835 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:49.834+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:50.153 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:50.152+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:50.885 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:50.884+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:51.153 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:51.152+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:51.837 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:51.836+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:52.193 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:52.192+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:52.818 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:52.817+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:53.193 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:53.192+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:53.782 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:53.782+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:54.212 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:54.210+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:54.827 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:54.825+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:55.251 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:55.249+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:55.833 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:55.831+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:56.236 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:56.234+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:56.855 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:56.854+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:57.252 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:57.250+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:57.895 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:57.893+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:58.241 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:58.238+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:58.927 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:58.925+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:59.205 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:03:59.203+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:03:59.888 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:03:59.886+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:00.191 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:00.189+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:00.903 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:00.901+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:01.151 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:01.149+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:01.895 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:01.893+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:02.174 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:02.172+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:02.939 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:02.937+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:03.142 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:03.141+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:03.953 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:03.951+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:04.115 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:04.113+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:04.923 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:04.921+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:05.135 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:05.133+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:05.935 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:05.933+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:06.114 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:06.112+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:06.946 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:06.944+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:07.123 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:07.121+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:07.903 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:07.901+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:08.076 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:08.074+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:08.884 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:08.882+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:09.110 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:09.108+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:09.924 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:09.922+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:10.095 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:10.093+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:10.937 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:10.935+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:11.104 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:11.102+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:11.898 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:11.896+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:12.081 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:12.079+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:12.947 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:12.945+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:13.065 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:13.063+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:13.967 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:13.965+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:14.111 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:14.109+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:14.979 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:14.977+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:15.106 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:15.104+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:16.005 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:16.003+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:16.106 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:16.104+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:17.055 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:17.053+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:17.075 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:17.073+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:18.057 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:18.055+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:18.118 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:18.116+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:19.021 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:19.019+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:19.141 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:19.139+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:19.987 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:19.985+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:20.144 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:20.142+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:20.967 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:20.965+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:21.175 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:21.173+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:21.971 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:21.969+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:22.153 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:22.151+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:22.985 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:22.983+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:23.171 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:23.169+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:23.993 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:23.991+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:24.221 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:24.219+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:25.011 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:25.008+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:25.183 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:25.181+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:25.999 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:25.997+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:26.232 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:26.230+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:26.999 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:26.997+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:27.192 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:27.190+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:28.003 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:28.001+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:28.147 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:28.145+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:28.980 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:28.978+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:29.178 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:29.176+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:30.017 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:30.015+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:30.168 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:30.166+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:30.981 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:30.979+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:31.123 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:31.121+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:31.938 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:31.937+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:32.082 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:32.081+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:32.921 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:32.920+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:33.090 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:33.088+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:33.972 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:33.970+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:34.096 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:34.095+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:34.999 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:34.998+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:35.070 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:35.069+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:36.041 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:36.039+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:36.055 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:36.054+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:37.072 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:37.071+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:37.088 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:37.087+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:38.073 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:38.072+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:38.117 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:38.116+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:39.072 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:39.071+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:39.101 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:39.100+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:40.088 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:40.087+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:40.127 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:40.126+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:41.100 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:41.099+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:41.148 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:41.147+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:42.105 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:42.104+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:42.150 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:42.149+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:43.132 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:43.131+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:43.185 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:43.184+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:44.105 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:44.104+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:44.161 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:44.160+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:45.150 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:45.148+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:45.157 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:45.157+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:46.130 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:46.129+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:46.205 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:46.204+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:47.134 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:47.133+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:47.228 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:47.227+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:48.137 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:48.136+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:48.202 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:48.201+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:49.091 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:49.090+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:49.177 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:49.176+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:50.134 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:50.133+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:50.191 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:50.190+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:51.183 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:51.182+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:51.183 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:51.183+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:52.216 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:52.214+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:52.217 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:52.216+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:53.235 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:53.233+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:53.245 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:53.244+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:54.188 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:54.186+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:54.206 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:54.204+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:55.174 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:55.172+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:55.217 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:55.215+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:56.183 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:56.181+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:56.229 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:56.227+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:57.144 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:57.142+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:57.274 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:57.272+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:58.099 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:58.097+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:58.268 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:58.266+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:59.137 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:04:59.135+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:04:59.262 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:04:59.260+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:00.160 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:00.158+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:00.290 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:00.288+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:01.201 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:01.199+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:01.332 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:01.330+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:02.218 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:02.216+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:02.329 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:02.327+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:03.250 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:03.248+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:03.318 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:03.316+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:04.220 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:04.218+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:04.279 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:04.277+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:05.205 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:05.203+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:05.253 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:05.251+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:06.176 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:06.174+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:06.289 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:06.287+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:07.178 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:07.176+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:07.270 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:07.268+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:08.132 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:08.130+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:08.256 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:08.254+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:09.153 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:09.150+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:09.234 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:09.232+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:10.187 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:10.185+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:10.271 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:10.269+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:11.238 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:11.236+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:11.242 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:11.241+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:12.217 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:12.215+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:12.244 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:12.242+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:13.203 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:13.201+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:13.207 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:13.205+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:14.234 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:14.231+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:14.240 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:14.238+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:15.228 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:15.226+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:15.288 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:15.286+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:16.261 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:16.258+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:16.311 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:16.309+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:17.243 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:17.241+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:17.304 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:17.302+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:18.246 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:18.245+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:18.318 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:18.316+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:19.270 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:19.265+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:19.351 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:19.349+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:20.278 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:20.276+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:20.360 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:20.358+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:21.309 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:21.308+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:21.333 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:21.332+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:22.325 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:22.323+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:22.348 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:22.346+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:23.304 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:23.302+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:23.317 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:23.315+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:24.274 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:24.272+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:24.308 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:24.306+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:25.315 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:25.313+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:25.342 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:25.340+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:26.356 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:26.354+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:26.360 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:26.358+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:27.319 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:27.317+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:27.404 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:27.402+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:28.294 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:28.289+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:28.379 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:28.377+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:29.259 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:29.257+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:29.397 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:29.395+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:30.269 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:30.268+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:30.409 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:30.408+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:31.312 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:31.311+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:31.414 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:31.413+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:32.274 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:32.273+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:32.436 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:32.435+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:33.257 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:33.256+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:33.403 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:33.402+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:34.289 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:34.287+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:34.429 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:34.428+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:35.317 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:35.316+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:35.411 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:35.410+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:36.366 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:36.365+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:36.413 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:36.412+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:37.393 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:37.392+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:37.420 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:37.419+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:38.396 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:38.395+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:38.451 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:38.450+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:39.426 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:39.425+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:39.441 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:39.440+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:40.405 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:40.403+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:40.463 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:40.462+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:41.392 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:41.391+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:41.479 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:41.478+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:42.360 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:42.359+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:42.510 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:42.509+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:43.323 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:43.322+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:43.464 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:43.463+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:44.283 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:44.282+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:44.465 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:44.464+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:45.300 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:45.299+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:45.431 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:45.430+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:46.257 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:46.256+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:46.423 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:46.422+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:47.216 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:47.215+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:47.468 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:47.467+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:48.218 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:48.217+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:48.431 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:48.430+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:49.215 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:49.214+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:49.479 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:49.478+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:50.188 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:50.187+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:50.453 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:50.452+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:51.203 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:51.201+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:51.464 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:51.462+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:52.179 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:52.177+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:52.465 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:52.463+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:53.145 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:53.142+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:53.431 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:53.429+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:54.121 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:54.119+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:54.393 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:54.391+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:55.152 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:55.150+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:55.420 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:55.418+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:56.144 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:56.142+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:56.454 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:56.452+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:57.114 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:57.112+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:57.476 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:57.474+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:58.118 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:58.117+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:58.504 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:58.503+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:59.159 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:05:59.157+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:05:59.501 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:05:59.499+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:00.193 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:00.191+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:00.502 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:00.500+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:01.228 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:01.226+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:01.477 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:01.475+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:02.204 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:02.202+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:02.499 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:02.497+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:03.200 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:03.198+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:03.549 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:03.547+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:04.201 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:04.199+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:04.501 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:04.499+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:05.164 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:05.161+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:05.525 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:05.523+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:06.206 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:06.204+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:06.495 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:06.493+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:07.246 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:07.244+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:07.515 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:07.513+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:08.239 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:08.237+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:08.478 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:08.476+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:09.213 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:09.211+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:09.513 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:09.511+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:10.166 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:10.164+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:10.470 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:10.468+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:11.181 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:11.179+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:11.455 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:11.453+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:12.180 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:12.179+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:12.407 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:12.405+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:13.203 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:13.201+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:13.446 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:13.444+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:14.227 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:14.225+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:14.473 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:14.471+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:15.241 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:15.239+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:15.521 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:15.519+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:16.291 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:16.289+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:16.495 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:16.493+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:17.319 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:17.317+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:17.530 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:17.528+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:18.325 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:18.323+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:18.547 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:18.545+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:19.278 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:19.276+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:19.590 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:19.588+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:20.270 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:20.268+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:20.634 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:20.632+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:21.248 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:21.246+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:21.641 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:21.639+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:22.273 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:22.271+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:22.678 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:22.676+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:23.317 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:23.315+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:23.651 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:23.649+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:24.281 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:24.279+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:24.655 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:24.653+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:25.277 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:25.275+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:25.664 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:25.662+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:26.234 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:26.232+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:26.706 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:26.704+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:27.242 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:27.241+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:27.674 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:27.672+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:28.221 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:28.219+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:28.688 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:28.687+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:29.194 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:29.193+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:29.670 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:29.669+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:30.154 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:30.153+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:30.672 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:30.671+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:31.158 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:31.156+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:31.652 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:31.651+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:32.152 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:32.151+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:32.655 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:32.654+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:33.184 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:33.183+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:33.620 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:33.619+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:34.154 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:34.153+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:34.592 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:34.591+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:35.158 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:35.157+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:35.593 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:35.592+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:36.154 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:36.153+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:36.589 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:36.588+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:37.202 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:37.202+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:37.636 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:37.635+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:38.199 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:38.198+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:38.681 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:38.680+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:39.238 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:39.237+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:39.632 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:39.631+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:40.285 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:40.284+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:40.619 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:40.618+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:41.320 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:41.319+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:41.611 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:41.611+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:42.368 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:42.366+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:42.624 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:42.623+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:43.363 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:43.362+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:43.618 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:43.617+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:44.359 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:44.358+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:44.652 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:44.651+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:45.311 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:45.310+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:45.641 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:45.639+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:46.317 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:46.316+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:46.679 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:46.678+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:47.291 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:47.290+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:47.689 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:47.688+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:48.255 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:48.254+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:48.692 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:48.691+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:49.269 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:49.267+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:49.686 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:49.684+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:50.237 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:50.235+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:50.675 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:50.673+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:51.224 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:51.222+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:51.630 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:51.628+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:52.272 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:52.270+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:52.599 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:52.597+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:53.236 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:53.234+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:53.626 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:53.624+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:54.276 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:54.274+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:54.656 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:54.654+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:55.310 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:55.308+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:55.670 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:55.668+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:56.307 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:56.305+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:56.634 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:56.632+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:57.328 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:57.326+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:57.602 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:57.600+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:58.298 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:58.296+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:58.594 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:58.592+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:59.264 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:06:59.262+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:06:59.561 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:06:59.559+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:00.294 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:00.292+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:00.571 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:00.569+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:01.248 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:01.246+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:01.592 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:01.590+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:02.255 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:02.253+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:02.557 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:02.555+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:03.257 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:03.255+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:03.588 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:03.586+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:04.272 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:04.270+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:04.570 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:04.568+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:05.277 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:05.276+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:05.557 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:05.555+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:06.298 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:06.296+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:06.561 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:06.559+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:07.334 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:07.332+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:07.603 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:07.601+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:08.323 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:08.321+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:08.638 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:08.637+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:09.324 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:09.322+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:09.677 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:09.675+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:10.358 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:10.356+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:10.645 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:10.643+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:11.333 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:11.331+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:11.598 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:11.596+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:12.306 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:12.304+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:12.602 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:12.600+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:13.351 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:13.349+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:13.631 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:13.629+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:14.326 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:14.324+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:14.664 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:14.662+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:15.304 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:15.302+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:15.626 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:15.624+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:16.345 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:16.343+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:16.640 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:16.638+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:17.336 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:17.334+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:17.610 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:17.608+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:18.321 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:18.319+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:18.604 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:18.602+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:19.287 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:19.285+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:19.600 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:19.598+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:20.330 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:20.328+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:20.598 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:20.596+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:21.284 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:21.282+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:21.587 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:21.585+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:22.316 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:22.314+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:22.569 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:22.567+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:23.341 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:23.339+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:23.538 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:23.536+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:24.304 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:24.302+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:24.563 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:24.561+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:25.276 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:25.274+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:25.525 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:25.523+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:26.259 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:26.255+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:26.545 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:26.543+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:27.238 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:27.237+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:27.503 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:27.502+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:28.237 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:28.236+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:28.491 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:28.490+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:29.254 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:29.253+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:29.529 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:29.528+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:30.297 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:30.296+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:30.500 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:30.499+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:31.285 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:31.284+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:31.461 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:31.460+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:32.321 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:32.320+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:32.414 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:32.413+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:33.367 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:33.366+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:33.386 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:33.385+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:34.354 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:34.353+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:34.361 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:34.360+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:35.326 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:35.325+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:35.399 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:35.398+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:36.294 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:36.292+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:36.433 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:36.432+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:37.288 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:37.287+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:37.397 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:37.396+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:38.293 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:38.292+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:38.352 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:38.351+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:39.292 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:39.291+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:39.386 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:39.385+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:40.312 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:40.311+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:40.391 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:40.390+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:41.300 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:41.299+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:41.348 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:41.347+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:42.299 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:42.298+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:42.334 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:42.333+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:43.309 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:43.308+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:43.317 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:43.316+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:44.316 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:44.315+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:44.336 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:44.335+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:45.291 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:45.290+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:45.324 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:45.323+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:46.299 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:46.298+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:46.318 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:46.317+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:47.288 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:47.287+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:47.291 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:47.290+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:48.273 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:48.271+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:48.303 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:48.301+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:49.279 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:49.277+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:49.341 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:49.339+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:50.244 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:50.242+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:50.303 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:50.301+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:51.241 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:51.239+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:51.314 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:51.313+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:52.219 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:52.217+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:52.342 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:52.341+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:53.228 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:53.226+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:53.376 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:53.374+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:54.259 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:54.257+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:54.369 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:54.367+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:55.301 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:55.299+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:55.400 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:55.398+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:56.349 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:56.347+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:56.389 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:56.387+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:57.372 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:57.370+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:57.422 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:57.420+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:58.373 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:58.371+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:58.399 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:58.397+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:59.356 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:07:59.354+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:07:59.378 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:07:59.376+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:00.335 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:00.333+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:00.398 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:00.396+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:01.301 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:01.299+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:01.405 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:01.403+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:02.270 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:02.268+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:02.437 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:02.435+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:03.280 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:03.278+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:03.414 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:03.412+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:04.241 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:04.239+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:04.403 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:04.401+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:05.245 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:05.243+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:05.435 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:05.433+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:06.215 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:06.213+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:06.465 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:06.463+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:07.171 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:07.169+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:07.510 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:07.509+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:08.127 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:08.125+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:08.544 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:08.542+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:09.162 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:09.160+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:09.563 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:09.561+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:10.147 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:10.145+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:10.597 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:10.595+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:11.159 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:11.157+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:11.575 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:11.573+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:12.124 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:12.123+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:12.569 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:12.567+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:13.125 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:13.123+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:13.526 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:13.524+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:14.139 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:14.137+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:14.530 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:14.528+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:15.112 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:15.110+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:15.523 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:15.521+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:16.118 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:16.117+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:16.528 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:16.526+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:17.128 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:17.126+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:17.561 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:17.559+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:18.168 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:18.166+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:18.588 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:18.586+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:19.121 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:19.119+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:19.571 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:19.569+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:20.078 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:20.077+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:20.580 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:20.578+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:21.041 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:21.039+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:21.594 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:21.592+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:22.085 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:22.083+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:22.594 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:22.592+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:23.093 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:23.090+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:23.598 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:23.596+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:24.124 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:24.123+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:24.620 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:24.618+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:25.139 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:25.138+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:25.650 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:25.649+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:26.172 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:26.171+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:26.602 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:26.601+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:27.195 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:27.194+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:27.644 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:27.643+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:28.159 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:28.158+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:28.620 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:28.619+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:29.155 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:29.154+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:29.632 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:29.631+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:30.175 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:30.175+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:30.655 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:30.654+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:31.214 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:31.212+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:31.702 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:31.701+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:32.188 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:32.187+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:32.701 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:32.700+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:33.157 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:33.155+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:33.719 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:33.718+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:34.164 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:34.163+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:34.716 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:34.715+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:35.196 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:35.195+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:35.685 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:35.684+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:36.187 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:36.186+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:36.706 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:36.706+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:37.151 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:37.150+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:37.705 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:37.704+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:38.122 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:38.121+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:38.664 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:38.663+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:39.110 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:39.109+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:39.679 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:39.678+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:40.101 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:40.100+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:40.639 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:40.638+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:41.053 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:41.052+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:41.630 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:41.629+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:42.026 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:42.025+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:42.625 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:42.624+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:43.075 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:43.074+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:43.601 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:43.600+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:44.063 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:44.062+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:44.651 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:44.650+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:45.025 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:45.023+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:45.689 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:45.688+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:46.031 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:46.029+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:46.647 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:46.645+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:47.009 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:47.007+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:47.629 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:47.627+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:48.007 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:48.005+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:48.637 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:48.635+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:48.966 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:48.964+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:49.602 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:49.600+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:50.013 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:50.011+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:50.568 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:50.566+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:51.049 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:51.047+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:51.527 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:51.525+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:52.032 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:52.030+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:52.569 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:52.567+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:53.032 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:53.030+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:53.599 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:53.597+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:53.997 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:53.995+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:54.554 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:54.552+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:54.958 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:54.957+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:55.563 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:55.561+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:55.920 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:55.918+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:56.609 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:56.607+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:56.945 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:56.943+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:57.609 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:57.607+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:57.919 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:57.917+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:58.635 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:58.633+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:58.940 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:58.938+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:59.592 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:08:59.590+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:08:59.946 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:08:59.944+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:00.591 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:00.589+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:00.977 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:00.975+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:01.577 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:01.575+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:01.972 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:01.970+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:02.564 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:02.562+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:02.926 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:02.923+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:03.613 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:03.611+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:03.895 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:03.893+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:04.596 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:04.594+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:04.860 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:04.858+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:05.623 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:05.621+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:05.834 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:05.833+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:06.647 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:06.645+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:06.840 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:06.838+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:07.660 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:07.658+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:07.850 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:07.848+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:08.653 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:08.651+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:08.891 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:08.889+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:09.627 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:09.625+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:09.934 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:09.932+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:10.651 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:10.649+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:10.929 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:10.927+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:11.693 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:11.691+0000 7ff685387700 -1 osd.0 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:11.896 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:11.895+0000 7f6cbabe8700 -1 osd.1 158 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:12.669 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:12.667+0000 7ff685387700 -1 osd.0 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:12.870 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:12.868+0000 7f6cbabe8700 -1 osd.1 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:13.630 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:13.629+0000 7ff685387700 -1 osd.0 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:13.913 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:13.911+0000 7f6cbabe8700 -1 osd.1 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:14.646 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:14.644+0000 7ff685387700 -1 osd.0 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:14.901 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:14.899+0000 7f6cbabe8700 -1 osd.1 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:15.651 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:15.649+0000 7ff685387700 -1 osd.0 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:15.915 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:15.913+0000 7f6cbabe8700 -1 osd.1 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:16.659 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:16.657+0000 7ff685387700 -1 osd.0 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:16.886 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:16.884+0000 7f6cbabe8700 -1 osd.1 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:17.703 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:17.701+0000 7ff685387700 -1 osd.0 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:17.901 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:17.899+0000 7f6cbabe8700 -1 osd.1 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:18.689 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:18.687+0000 7ff685387700 -1 osd.0 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:18.878 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:18.876+0000 7f6cbabe8700 -1 osd.1 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:19.714 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:19.712+0000 7ff685387700 -1 osd.0 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:19.836 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:19.835+0000 7f6cbabe8700 -1 osd.1 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:20.743 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:20.741+0000 7ff685387700 -1 osd.0 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:20.833 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:20.831+0000 7f6cbabe8700 -1 osd.1 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:21.699 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:21.697+0000 7ff685387700 -1 osd.0 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:21.796 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:21.794+0000 7f6cbabe8700 -1 osd.1 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:22.728 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:22.726+0000 7ff685387700 -1 osd.0 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:22.756 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:22.754+0000 7f6cbabe8700 -1 osd.1 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:23.715 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:23.715+0000 7ff685387700 -1 osd.0 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:23.796 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:23.795+0000 7f6cbabe8700 -1 osd.1 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:24.747 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:24.746+0000 7f6cbabe8700 -1 osd.1 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:24.761 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:24.760+0000 7ff685387700 -1 osd.0 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:25.723 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:25.722+0000 7ff685387700 -1 osd.0 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:25.797 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:25.796+0000 7f6cbabe8700 -1 osd.1 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:26.691 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:26.690+0000 7ff685387700 -1 osd.0 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:26.830 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:26.829+0000 7f6cbabe8700 -1 osd.1 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:27.643 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:27.642+0000 7ff685387700 -1 osd.0 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:27.838 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:27.837+0000 7f6cbabe8700 -1 osd.1 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:28.677 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:28.676+0000 7ff685387700 -1 osd.0 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:28.855 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:28.854+0000 7f6cbabe8700 -1 osd.1 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:29.697 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:29.696+0000 7ff685387700 -1 osd.0 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:29.902 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:29.901+0000 7f6cbabe8700 -1 osd.1 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:30.713 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:30.713+0000 7ff685387700 -1 osd.0 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:30.885 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:30.884+0000 7f6cbabe8700 -1 osd.1 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:31.667 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:31.666+0000 7ff685387700 -1 osd.0 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:31.893 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:31.892+0000 7f6cbabe8700 -1 osd.1 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:32.660 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:32.659+0000 7ff685387700 -1 osd.0 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:32.896 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:32.895+0000 7f6cbabe8700 -1 osd.1 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:33.697 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:33.696+0000 7ff685387700 -1 osd.0 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:33.934 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:33.933+0000 7f6cbabe8700 -1 osd.1 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:34.741 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:34.740+0000 7ff685387700 -1 osd.0 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:34.951 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:34.950+0000 7f6cbabe8700 -1 osd.1 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:35.743 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:35.742+0000 7ff685387700 -1 osd.0 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:35.988 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:35.987+0000 7f6cbabe8700 -1 osd.1 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:36.785 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:36.784+0000 7ff685387700 -1 osd.0 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:37.037 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:37.036+0000 7f6cbabe8700 -1 osd.1 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:37.788 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:37.787+0000 7ff685387700 -1 osd.0 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:38.007 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:38.006+0000 7f6cbabe8700 -1 osd.1 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:38.818 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:38.817+0000 7ff685387700 -1 osd.0 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:39.032 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:39.031+0000 7f6cbabe8700 -1 osd.1 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:39.843 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:39.842+0000 7ff685387700 -1 osd.0 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:40.007 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:40.006+0000 7f6cbabe8700 -1 osd.1 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:40.862 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:40.861+0000 7ff685387700 -1 osd.0 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:41.006 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:41.006+0000 7f6cbabe8700 -1 osd.1 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:41.869 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:41.867+0000 7ff685387700 -1 osd.0 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:41.999 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:41.998+0000 7f6cbabe8700 -1 osd.1 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:42.889 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:42.888+0000 7ff685387700 -1 osd.0 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:43.011 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:43.010+0000 7f6cbabe8700 -1 osd.1 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:43.931 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:43.929+0000 7ff685387700 -1 osd.0 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:44.054 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:44.052+0000 7f6cbabe8700 -1 osd.1 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:44.944 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:44.942+0000 7ff685387700 -1 osd.0 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:45.043 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:45.041+0000 7f6cbabe8700 -1 osd.1 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:45.955 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:45.953+0000 7ff685387700 -1 osd.0 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:46.013 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:46.011+0000 7f6cbabe8700 -1 osd.1 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:46.970 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:46.968+0000 7ff685387700 -1 osd.0 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:46.985 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:46.983+0000 7f6cbabe8700 -1 osd.1 159 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:47.972 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:47.970+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:47.986 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:47.984+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:48.964 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:48.962+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:48.986 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:48.984+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:49.930 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:49.928+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:49.966 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:49.964+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:50.886 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:50.884+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:50.955 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:50.954+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:51.926 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:51.924+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:51.960 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:51.958+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:52.886 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:52.884+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:52.956 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:52.954+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:53.925 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:53.923+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:53.984 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:53.982+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:54.944 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:54.942+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:55.020 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:54.956+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:55.920 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:55.918+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:55.994 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:55.992+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:56.908 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:56.906+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:57.029 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:57.027+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:57.935 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:57.933+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:58.013 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:58.011+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:58.970 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:58.967+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:59.041 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:09:59.038+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:09:59.924 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:09:59.922+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:00.037 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:00.035+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:00.896 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:00.894+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:01.025 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:01.023+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:01.869 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:01.867+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:02.011 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:02.009+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:02.856 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:02.854+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:03.052 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:03.050+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:03.829 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:03.827+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:04.041 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:04.039+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:04.827 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:04.825+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:05.074 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:05.072+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:05.778 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:05.777+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:06.034 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:06.032+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:06.802 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:06.800+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:07.044 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:07.042+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:07.763 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:07.761+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:08.054 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:08.052+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:08.786 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:08.784+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:09.087 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:09.085+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:09.785 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:09.783+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:10.073 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:10.071+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:10.823 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:10.821+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:11.103 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:11.101+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:11.785 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:11.783+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:12.068 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:12.066+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:12.751 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:12.749+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:13.080 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:13.078+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:13.753 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:13.751+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:14.056 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:14.054+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:14.769 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:14.767+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:15.025 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:15.023+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:15.744 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:15.742+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:16.041 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:16.039+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:16.751 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:16.749+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:17.067 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:17.065+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:17.759 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:17.757+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:18.064 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:18.062+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:18.756 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:18.754+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:19.027 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:19.025+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:19.708 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:19.706+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:20.018 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:20.016+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:20.740 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:20.738+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:20.982 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:20.981+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:21.743 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:21.742+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:22.014 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:22.013+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:22.717 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:22.716+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:23.062 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:23.061+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:23.713 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:23.712+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:24.033 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:24.032+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:24.666 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:24.665+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:25.024 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:25.023+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:25.709 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:25.708+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:26.014 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:26.013+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:26.683 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:26.682+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:27.034 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:27.033+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:27.635 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:27.634+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:27.986 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:27.985+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:28.651 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:28.650+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:29.031 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:29.030+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:29.639 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:29.638+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:30.018 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:30.017+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:30.606 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:30.605+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:30.982 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:30.981+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:31.567 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:31.566+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:31.950 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:31.949+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:32.603 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:32.602+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:32.920 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:32.919+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:33.649 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:33.648+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:33.873 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:33.872+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:34.685 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:34.684+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:34.899 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:34.898+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:35.703 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:35.702+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:35.885 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:35.884+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:36.689 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:36.688+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:36.920 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:36.919+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:37.657 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:37.655+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:37.963 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:37.962+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:38.630 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:38.629+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:38.989 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:38.988+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:39.601 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:39.600+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:39.945 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:39.944+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:40.601 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:40.600+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:40.912 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:40.912+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:41.577 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:41.576+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:41.865 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:41.863+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:42.609 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:42.607+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:42.881 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:42.879+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:43.603 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:43.601+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:43.860 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:43.858+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:44.629 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:44.627+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:44.898 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:44.896+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:45.600 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:45.598+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:45.862 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:45.861+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:46.608 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:46.606+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:46.899 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:46.897+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:47.613 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:47.611+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:47.933 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:47.931+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:48.633 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:48.631+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:48.924 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:48.917+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:49.669 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:49.667+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:49.921 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:49.919+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:50.635 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:50.633+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:50.940 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:50.938+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:51.662 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:51.660+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:51.965 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:51.963+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:52.669 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:52.667+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:52.950 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:52.949+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:53.670 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:53.668+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:53.953 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:53.951+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:54.681 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:54.679+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:54.962 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:54.960+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:55.661 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:55.659+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:55.931 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:55.929+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:56.669 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:56.667+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:56.898 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:56.896+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:57.714 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:57.712+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:57.871 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:57.869+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:58.691 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:58.689+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:58.893 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:58.891+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:59.678 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:10:59.676+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:10:59.930 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:10:59.928+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:00.728 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:00.726+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:00.907 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:00.905+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:01.694 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:01.692+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:01.949 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:01.946+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:02.701 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:02.699+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:02.933 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:02.931+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:03.694 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:03.692+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:03.965 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:03.963+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:04.716 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:04.714+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:04.992 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:04.990+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:05.746 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:05.744+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:05.963 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:05.961+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:06.777 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:06.775+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:07.010 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:07.008+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:07.815 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:07.813+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:08.021 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:08.019+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:08.820 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:08.818+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:08.981 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:08.979+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:09.793 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:09.791+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:09.971 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:09.969+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:10.835 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:10.833+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:11.016 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:11.014+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:11.792 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:11.790+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:12.008 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:12.007+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:12.757 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:12.755+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:13.005 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:13.003+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:13.726 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:13.724+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:14.048 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:14.046+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:14.701 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:14.699+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:15.057 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:15.054+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:15.693 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:15.691+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:16.098 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:16.096+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:16.736 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:16.734+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:17.131 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:17.129+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:17.779 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:17.777+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:18.109 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:18.107+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:18.756 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:18.754+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:19.123 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:19.121+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:19.764 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:19.763+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:20.099 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:20.098+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:20.782 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:20.782+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:21.099 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:21.098+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:21.738 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:21.737+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:22.148 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:22.147+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:22.758 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:22.757+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:23.111 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:23.110+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:23.807 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:23.806+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:24.151 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:24.150+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:24.783 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:24.782+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:25.171 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:25.170+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:25.795 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:25.794+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:26.162 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:26.161+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:26.806 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:26.805+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:27.183 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:27.182+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:27.804 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:27.803+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:28.173 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:28.172+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:28.775 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:28.774+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:29.191 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:29.190+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:29.762 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:29.761+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:30.234 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:30.233+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:30.798 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:30.797+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:31.198 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:31.197+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:31.763 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:31.762+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:32.213 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:32.212+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:32.724 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:32.723+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:33.223 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:33.222+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:33.770 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:33.769+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:34.187 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:34.186+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:34.787 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:34.786+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:35.182 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:35.181+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:35.752 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:35.751+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:36.190 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:36.189+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:36.753 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:36.752+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:37.213 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:37.212+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:37.787 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:37.786+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:38.195 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:38.194+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:38.822 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:38.821+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:39.239 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:39.238+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:39.814 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:39.812+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:40.195 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:40.193+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:40.837 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:40.835+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:41.150 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:41.148+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:41.867 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:41.865+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:42.110 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:42.108+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:42.873 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:42.871+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:43.102 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:43.100+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:43.881 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:43.879+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:44.069 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:44.067+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:44.887 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:44.885+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:45.043 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:45.041+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:45.891 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:45.889+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:46.072 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:46.070+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:46.916 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:46.914+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:47.061 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:47.059+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:47.937 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:47.935+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:48.102 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:48.100+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:48.984 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:48.982+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:49.082 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:49.080+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:49.998 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:49.996+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:50.073 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:50.071+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:50.963 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:50.961+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:51.104 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:51.102+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:51.962 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:51.960+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:52.083 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:52.081+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:52.987 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:52.985+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:53.055 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:53.053+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:53.947 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:53.945+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:54.011 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:54.009+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:54.924 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:54.922+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:55.029 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:55.027+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:55.927 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:55.925+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:56.017 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:56.015+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:56.922 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:56.920+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:56.990 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:56.988+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:57.917 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:57.915+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:57.977 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:57.975+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:58.871 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:58.868+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:58.945 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:58.943+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:59.879 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:11:59.878+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:11:59.926 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:11:59.925+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:00.835 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:00.833+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:00.950 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:00.948+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:01.862 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:01.860+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:01.950 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:01.949+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:02.815 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:02.813+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:02.907 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:02.905+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:03.842 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:03.840+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:03.898 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:03.896+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:04.805 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:04.803+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:04.929 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:04.927+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:05.768 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:05.766+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:05.947 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:05.945+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:06.758 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:06.756+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:06.906 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:06.905+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:07.791 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:07.789+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:07.910 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:07.908+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:08.822 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:08.820+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:08.916 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:08.914+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:09.819 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:09.817+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:09.878 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:09.877+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:10.786 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:10.784+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:10.836 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:10.833+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:11.781 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:11.779+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:11.825 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:11.823+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:12.779 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:12.777+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:12.821 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:12.819+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:13.765 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:13.763+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:13.799 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:13.797+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:14.722 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:14.720+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:14.776 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:14.775+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:15.731 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:15.729+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:15.791 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:15.789+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:16.743 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:16.741+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:16.769 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:16.767+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:17.757 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:17.756+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:17.793 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:17.792+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:18.743 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:18.742+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:18.779 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:18.778+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:19.737 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:19.736+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:19.748 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:19.747+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:20.715 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:20.714+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:20.764 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:20.763+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:21.685 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:21.684+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:21.807 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:21.806+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:22.684 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:22.683+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:22.803 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:22.802+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:23.651 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:23.650+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:23.798 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:23.797+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:24.683 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:24.682+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:24.755 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:24.754+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:25.724 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:25.723+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:25.756 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:25.755+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:26.681 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:26.679+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:26.791 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:26.790+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:27.699 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:27.698+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:27.781 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:27.780+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:28.659 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:28.658+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:28.786 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:28.785+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:29.676 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:29.676+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:29.798 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:29.797+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:30.635 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:30.634+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:30.801 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:30.800+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:31.662 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:31.661+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:31.776 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:31.775+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:32.705 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:32.704+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:32.732 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:32.731+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:33.685 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:33.684+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:33.699 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:33.698+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:34.657 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:34.656+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:34.662 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:34.661+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:35.639 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:35.638+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:35.708 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:35.707+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:36.637 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:36.636+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:36.753 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:36.752+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:37.686 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:37.684+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:37.746 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:37.744+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:38.649 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:38.647+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:38.714 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:38.712+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:39.611 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:39.609+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:39.679 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:39.677+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:40.639 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:40.637+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:40.676 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:40.674+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:41.626 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:41.624+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:41.628 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:41.627+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:42.603 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:42.601+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:42.649 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:42.647+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:43.626 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:43.625+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:43.641 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:43.639+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:44.645 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:44.643+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:44.663 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:44.661+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:45.636 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:45.634+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:45.652 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:45.651+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:46.645 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:46.643+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:46.675 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:46.673+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:47.642 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:47.640+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:47.687 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:47.685+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:48.654 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:48.652+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:48.681 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:48.679+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:49.681 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:49.679+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:49.728 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:49.726+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:50.651 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:50.649+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:50.735 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:50.733+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:51.629 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:51.627+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:51.745 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:51.743+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:52.603 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:52.601+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:52.783 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:52.781+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:53.585 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:53.583+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:53.826 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:53.824+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:54.587 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:54.585+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:54.875 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:54.873+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:55.625 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:55.622+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:55.893 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:55.891+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:56.665 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:56.663+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:56.844 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:56.842+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:57.651 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:57.649+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:57.889 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:57.887+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:58.642 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:58.640+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:58.930 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:58.927+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:59.637 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:12:59.635+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:12:59.949 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:12:59.947+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:00.625 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:00.623+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:00.984 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:00.982+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:01.584 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:01.581+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:01.959 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:01.957+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:02.627 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:02.625+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:02.949 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:02.947+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:03.646 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:03.644+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:03.986 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:03.984+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:04.647 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:04.645+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:04.975 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:04.973+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:05.614 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:05.612+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:05.953 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:05.951+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:06.656 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:06.654+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:06.999 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:06.997+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:07.663 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:07.661+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:08.008 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:08.006+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:08.630 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:08.626+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:09.017 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:09.014+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:09.608 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:09.606+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:10.022 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:10.020+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:10.650 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:10.648+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:10.980 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:10.978+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:11.680 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:11.678+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:11.999 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:11.997+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:12.660 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:12.658+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:13.016 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:13.014+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:13.673 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:13.671+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:13.980 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:13.978+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:14.633 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:14.631+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:14.939 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:14.938+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:15.637 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:15.636+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:15.971 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:15.970+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:16.640 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:16.639+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:17.011 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:17.010+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:17.623 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:17.622+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:18.051 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:18.050+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:18.647 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:18.646+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:19.036 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:19.035+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:19.635 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:19.634+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:20.057 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:20.056+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:20.599 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:20.598+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:21.072 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:21.071+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:21.590 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:21.589+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:22.095 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:22.094+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:22.607 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:22.606+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:23.071 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:23.070+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:23.584 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:23.583+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:24.071 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:24.070+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:24.634 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:24.633+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:25.091 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:25.090+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:25.606 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:25.605+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:26.074 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:26.072+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:26.591 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:26.590+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:27.029 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:27.028+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:27.574 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:27.573+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:28.031 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:28.030+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:28.596 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:28.595+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:28.993 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:28.992+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:29.590 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:29.589+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:30.026 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:30.025+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:30.596 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:30.595+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:31.045 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:31.044+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:31.637 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:31.636+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:32.089 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:32.088+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:32.683 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:32.682+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:33.089 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:33.088+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:33.729 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:33.728+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:34.122 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:34.121+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:34.684 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:34.683+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:35.170 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:35.168+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:35.656 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:35.654+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:36.194 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:36.192+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:36.614 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:36.612+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:37.221 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:37.219+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:37.655 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:37.653+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:38.244 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:38.242+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:38.642 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:38.640+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:39.252 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:39.250+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:39.640 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:39.638+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:40.249 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:40.247+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:40.687 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:40.685+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:41.238 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:41.236+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:41.662 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:41.660+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:42.189 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:42.187+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:42.625 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:42.623+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:43.191 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:43.189+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:43.599 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:43.597+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:44.234 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:44.232+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:44.647 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:44.645+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:45.268 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:45.266+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:45.604 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:45.602+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:46.226 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:46.224+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:46.639 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:46.637+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:47.196 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:47.194+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:47.640 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:47.638+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:48.179 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:48.177+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:48.683 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:48.681+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:49.138 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:49.136+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:49.641 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:49.639+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:50.093 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:50.091+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:50.639 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:50.637+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:51.092 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:51.090+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:51.647 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:51.645+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:52.089 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:52.087+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:52.599 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:52.597+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:53.097 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:53.095+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:53.569 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:53.567+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:54.066 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:54.064+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:54.521 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:54.519+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:55.086 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:55.083+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:55.539 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:55.537+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:56.087 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:56.085+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:56.529 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:56.527+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:57.088 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:57.086+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:57.530 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:57.528+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:58.112 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:58.110+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:58.533 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:58.531+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:59.140 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:13:59.138+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:13:59.503 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:13:59.501+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:00.182 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:00.180+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:00.528 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:00.526+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:01.170 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:01.168+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:01.515 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:01.513+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:02.203 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:02.201+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:02.511 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:02.509+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:03.170 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:03.168+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:03.482 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:03.480+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:04.211 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:04.209+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:04.479 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:04.477+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:05.185 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:05.183+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:05.478 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:05.476+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:06.188 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:06.186+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:06.471 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:06.469+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:07.175 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:07.173+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:07.497 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:07.495+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:08.192 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:08.190+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:08.462 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:08.460+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:09.208 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:09.206+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:09.472 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:09.470+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:10.215 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:10.213+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:10.445 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:10.443+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:11.200 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:11.198+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:11.485 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:11.483+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:12.240 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:12.238+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:12.528 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:12.527+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:13.197 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:13.196+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:13.566 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:13.564+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:14.176 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:14.175+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:14.524 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:14.523+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:15.196 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:15.195+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:15.534 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:15.533+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:16.194 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:16.193+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:16.581 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:16.579+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:17.195 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:17.194+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:17.552 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:17.550+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:18.238 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:18.237+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:18.583 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:18.582+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:19.204 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:19.203+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:19.587 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:19.586+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:20.159 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:20.158+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:20.553 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:20.552+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:21.157 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:21.156+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:21.556 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:21.555+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:22.141 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:22.140+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:22.553 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:22.552+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:23.150 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:23.149+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:23.592 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:23.591+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:24.106 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:24.105+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:24.628 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:24.626+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:25.074 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:25.073+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:25.583 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:25.582+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:26.105 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:26.104+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:26.540 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:26.539+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:27.437 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:27.154+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:27.564 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:27.563+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:28.193 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:28.192+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:28.529 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:28.528+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:29.187 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:29.186+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:29.570 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:29.569+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:30.172 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:30.170+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:30.565 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:30.564+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:31.148 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:31.147+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:31.532 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:31.531+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:32.107 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:32.106+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:32.486 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:32.483+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:33.117 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:33.115+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:33.480 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:33.478+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:34.128 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:34.126+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:34.507 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:34.505+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:35.136 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:35.133+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:35.515 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:35.513+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:36.137 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:36.135+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:36.515 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:36.513+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:37.166 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:37.164+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:37.495 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:37.493+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:38.201 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:38.199+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:38.531 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:38.529+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:39.193 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:39.191+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:39.499 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:39.497+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:40.151 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:40.149+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:40.450 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:40.448+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:41.195 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:41.193+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:41.439 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:41.437+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:42.171 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:42.169+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:42.425 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:42.423+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:43.157 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:43.155+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:43.443 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:43.441+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:44.178 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:44.176+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:44.396 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:44.394+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:45.196 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:45.194+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:45.412 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:45.410+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:46.239 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:46.237+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:46.423 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:46.421+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:47.248 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:47.246+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:47.471 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:47.469+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:48.261 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:48.259+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:48.489 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:48.487+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:49.226 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:49.223+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:49.464 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:49.462+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:50.250 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:50.248+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:50.442 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:50.440+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:51.257 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:51.254+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:51.444 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:51.442+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:52.217 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:52.215+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:52.414 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:52.412+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:53.206 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:53.204+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:53.386 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:53.384+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:54.244 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:54.242+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:54.410 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:54.408+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:55.243 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:55.241+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:55.399 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:55.396+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:56.280 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:56.278+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:56.376 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:56.374+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:57.318 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:57.316+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:57.417 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:57.415+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:58.337 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:58.335+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:58.384 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:58.382+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:59.334 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:14:59.332+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:14:59.397 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:14:59.395+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:00.358 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:00.356+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:00.437 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:00.435+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:01.362 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:01.360+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:01.469 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:01.467+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:02.336 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:02.334+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:02.450 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:02.448+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:03.320 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:03.318+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:03.427 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:03.425+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:04.297 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:04.295+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:04.415 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:04.412+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:05.318 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:05.316+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:05.369 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:05.367+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:06.274 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:06.272+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:06.355 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:06.353+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:07.245 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:07.243+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:07.399 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:07.397+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:08.238 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:08.236+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:08.440 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:08.438+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:09.217 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:09.215+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:09.483 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:09.481+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:10.188 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:10.187+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:10.530 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:10.529+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:11.220 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:11.219+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:11.560 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:11.559+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:12.172 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:12.171+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:12.558 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:12.557+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:13.197 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:13.196+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:13.521 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:13.520+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:14.214 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:14.213+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:14.571 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:14.570+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:15.221 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:15.220+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:15.560 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:15.559+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:16.237 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:16.236+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:16.566 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:16.565+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:17.210 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:17.209+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:17.583 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:17.582+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:18.228 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:18.227+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:18.624 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:18.623+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:19.201 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:19.200+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:19.634 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:19.633+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:20.181 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:20.180+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:20.626 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:20.625+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:21.145 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:21.144+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:21.620 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:21.619+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:22.099 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:22.098+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:22.613 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:22.612+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:23.050 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:23.049+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:23.662 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:23.659+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:24.075 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:24.073+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:24.699 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:24.695+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:25.066 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:25.065+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:25.650 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:25.649+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:26.090 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:26.018+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:26.639 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:26.638+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:27.018 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:27.017+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:27.675 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:27.674+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:28.028 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:28.027+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:28.717 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:28.716+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:29.000 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:28.998+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:29.731 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:29.730+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:30.042 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:30.041+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:30.768 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:30.766+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:30.995 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:30.993+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:31.719 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:31.717+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:31.979 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:31.977+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:32.696 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:32.694+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:32.974 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:32.972+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:33.676 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:33.674+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:34.008 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:34.006+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:34.711 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:34.709+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:35.016 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:35.014+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:35.734 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:35.732+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:35.998 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:35.996+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:36.747 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:36.745+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:36.952 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:36.950+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:37.734 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:37.732+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:37.915 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:37.913+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:38.712 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:38.710+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:38.952 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:38.949+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:39.715 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:39.713+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:39.931 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:39.929+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:40.742 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:40.740+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:40.925 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:40.923+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:41.724 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:41.722+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:41.893 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:41.891+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:42.715 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:42.713+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:42.881 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:42.879+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:43.764 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:43.762+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:43.861 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:43.859+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:44.725 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:44.723+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:44.834 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:44.832+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:45.710 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:45.708+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:45.860 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:45.857+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:46.734 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:46.732+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:46.872 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:46.870+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:47.720 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:47.718+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:47.909 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:47.906+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:48.676 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:48.674+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:48.884 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:48.882+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:49.698 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:49.696+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:49.934 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:49.932+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:50.728 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:50.726+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:50.922 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:50.920+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:51.722 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:51.720+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:51.922 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:51.920+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:52.756 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:52.754+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:52.894 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:52.892+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:53.722 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:53.719+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:53.920 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:53.918+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:54.682 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:54.680+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:54.949 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:54.947+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:55.707 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:55.705+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:55.961 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:55.959+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:56.726 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:56.724+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:56.952 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:56.949+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:57.759 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:57.757+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:57.956 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:57.954+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:58.779 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:58.777+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:58.919 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:58.917+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:59.749 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:15:59.746+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:15:59.925 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:15:59.922+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:00.743 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:00.740+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:00.934 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:00.932+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:01.788 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:01.786+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:01.917 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:01.915+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:02.752 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:02.749+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:02.951 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:02.950+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:03.784 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:03.782+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:04.528 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:03.960+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:04.824 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:04.822+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:05.008 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:05.006+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:05.822 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:05.820+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:05.997 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:05.995+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:06.866 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:06.864+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:07.001 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:06.999+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:07.892 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:07.891+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:07.954 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:07.953+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:08.913 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:08.912+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:08.918 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:08.917+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:09.898 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:09.897+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:09.942 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:09.941+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:10.911 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:10.910+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:10.955 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:10.954+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:11.903 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:11.902+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:11.910 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:11.909+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:12.902 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:12.901+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:12.923 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:12.922+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:13.918 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:13.916+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:13.922 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:13.921+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:14.951 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:14.950+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:14.969 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:14.968+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:15.951 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:15.950+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:15.973 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:15.972+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:16.905 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:16.904+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:16.930 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:16.930+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:17.893 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:17.891+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:17.981 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:17.980+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:18.926 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:18.924+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:18.998 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:18.996+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:19.960 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:19.959+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:19.967 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:19.967+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:20.918 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:20.917+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:20.942 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:20.942+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:21.881 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:21.880+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:21.950 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:21.949+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:22.882 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:22.881+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:22.975 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:22.974+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:23.844 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:23.843+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:24.011 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:24.010+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:24.814 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:24.813+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:25.033 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:25.032+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:25.857 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:25.855+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:26.058 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:26.057+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:26.832 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:26.831+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:27.044 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:27.043+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:27.857 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:27.856+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:28.054 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:28.052+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:28.907 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:28.905+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:29.031 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:29.029+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:29.937 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:29.935+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:30.045 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:30.043+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:30.901 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:30.899+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:31.085 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:31.083+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:31.859 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:31.857+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:32.082 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:32.080+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:32.852 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:32.850+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:33.063 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:33.061+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:33.895 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:33.893+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:34.111 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:34.109+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:34.920 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:34.918+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:35.075 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:35.073+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:35.942 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:35.940+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:36.057 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:36.055+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:36.962 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:36.960+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:37.089 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:37.087+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:37.927 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:37.925+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:38.123 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:38.121+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:38.887 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:38.885+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:39.101 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:39.099+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:39.867 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:39.865+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:40.092 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:40.090+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:40.884 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:40.843+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:41.089 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:41.087+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:41.843 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:41.841+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:42.136 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:42.134+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:42.869 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:42.867+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:43.180 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:43.178+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:43.905 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:43.903+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:44.199 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:44.197+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:44.913 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:44.910+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:45.219 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:45.217+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:45.900 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:45.898+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:46.201 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:46.199+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:46.877 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:46.875+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:47.168 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:47.166+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:47.890 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:47.888+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:48.120 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:48.118+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:48.898 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:48.895+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:49.151 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:49.149+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:49.897 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:49.895+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:50.200 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:50.198+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:50.941 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:50.939+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:51.154 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:51.151+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:51.903 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:51.901+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:52.124 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:52.122+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:52.913 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:52.911+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:53.093 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:53.091+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:53.959 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:53.957+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:54.105 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:54.103+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:54.942 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:54.940+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:55.059 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:55.057+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:55.940 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:55.938+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:56.108 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:56.106+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:56.894 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:56.892+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:57.128 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:57.126+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:57.923 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:57.921+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:58.135 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:58.133+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:58.900 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:58.898+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:59.159 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:16:59.157+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:16:59.947 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:16:59.945+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:00.137 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:00.135+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:00.924 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:00.922+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:01.108 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:01.106+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:01.940 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:01.938+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:02.126 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:02.125+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:02.898 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:02.896+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:03.165 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:03.163+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:03.882 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:03.880+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:04.163 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:04.161+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:04.856 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:04.854+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:05.119 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:05.117+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:05.818 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:05.817+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:06.123 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:06.122+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:06.867 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:06.866+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:07.150 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:07.149+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:07.861 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:07.860+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:08.115 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:08.114+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:08.841 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:08.840+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:09.161 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:09.160+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:09.823 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:09.822+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:10.171 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:10.170+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:10.781 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:10.780+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:11.147 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:11.146+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:11.753 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:11.752+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:12.123 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:12.122+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:12.785 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:12.784+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:13.106 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:13.105+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:13.796 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:13.795+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:14.157 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:14.156+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:14.805 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:14.804+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:15.140 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:15.139+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:15.796 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:15.795+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:16.179 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:16.178+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:16.777 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:16.776+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:17.151 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:17.149+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:17.794 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:17.793+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:18.133 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:18.132+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:18.780 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:18.779+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:19.106 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:19.105+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:19.768 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:19.767+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:20.127 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:20.126+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:20.791 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:20.790+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:21.090 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:21.089+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:21.824 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:21.822+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:22.064 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:22.063+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:22.808 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:22.807+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:23.097 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:23.096+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:23.816 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:23.815+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:24.137 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:24.136+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:24.809 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:24.808+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:25.163 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:25.161+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:25.829 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:25.828+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:26.149 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:26.147+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:26.781 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:26.779+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:27.131 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:27.129+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:27.815 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:27.812+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:28.143 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:28.141+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:28.815 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:28.813+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:29.140 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:29.138+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:29.774 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:29.772+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:30.121 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:30.119+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:30.734 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:30.732+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:31.153 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:31.150+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:31.745 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:31.743+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:32.197 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:32.195+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:32.717 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:32.715+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:33.185 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:33.183+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:33.763 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:33.761+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:34.151 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:34.149+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:34.728 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:34.726+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:35.157 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:35.154+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:35.712 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:35.710+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:36.143 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:36.141+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:36.714 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:36.712+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:37.120 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:37.118+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:37.694 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:37.692+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:38.082 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:38.080+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:38.692 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:38.690+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:39.107 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:39.105+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:39.703 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:39.701+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:40.124 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:40.122+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:40.743 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:40.741+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:41.083 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:41.081+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:41.764 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:41.762+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:42.045 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:42.043+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:42.768 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:42.766+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:43.089 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:43.087+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:43.753 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:43.751+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:44.114 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:44.112+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:44.798 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:44.796+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:45.121 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:45.119+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:45.775 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:45.773+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:46.094 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:46.092+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:46.792 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:46.790+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:47.054 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:47.052+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:47.827 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:47.825+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:48.100 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:48.098+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:48.815 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:48.813+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:49.065 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:49.063+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:49.786 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:49.784+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:50.091 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:50.089+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:50.740 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:50.738+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:51.068 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:51.066+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:51.724 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:51.722+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:52.095 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:52.093+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:52.765 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:52.763+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:53.046 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:53.044+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:53.764 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:53.762+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:54.067 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:54.065+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:54.730 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:54.727+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:55.080 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:55.078+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:55.779 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:55.777+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:56.117 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:56.115+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:56.771 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:56.769+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:57.135 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:57.133+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:57.730 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:57.728+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:58.183 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:58.181+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:58.717 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:58.715+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:59.164 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:17:59.162+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:17:59.752 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:17:59.750+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:00.115 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:00.113+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:00.715 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:00.713+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:01.082 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:01.080+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:01.726 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:01.724+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:02.055 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:02.053+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:02.693 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:02.691+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:03.048 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:03.046+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:03.722 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:03.719+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:04.088 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:04.087+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:04.724 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:04.723+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:05.116 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:05.114+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:05.698 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:05.697+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:06.134 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:06.133+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:06.719 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:06.718+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:07.139 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:07.138+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:07.752 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:07.751+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:08.156 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:08.155+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:08.723 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:08.722+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:09.204 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:09.203+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:09.694 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:09.693+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:10.206 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:10.205+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:10.735 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:10.733+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:11.223 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:11.222+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:11.744 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:11.743+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:12.271 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:12.270+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:12.705 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:12.704+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:13.252 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:13.250+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:13.670 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:13.669+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:14.295 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:14.294+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:14.673 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:14.672+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:15.254 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:15.253+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:15.719 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:15.718+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:16.275 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:16.273+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:16.704 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:16.703+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:17.319 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:17.318+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:17.753 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:17.752+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:18.348 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:18.347+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:18.729 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:18.728+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:19.314 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:19.313+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:19.729 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:19.727+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:20.318 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:20.317+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:20.777 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:20.776+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:21.341 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:21.340+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:21.776 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:21.775+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:22.304 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:22.303+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:22.779 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:22.778+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:23.269 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:23.268+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:23.773 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:23.772+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:24.279 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:24.278+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:24.740 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:24.738+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:25.315 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:25.313+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:25.730 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:25.728+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:26.348 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:26.346+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:26.778 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:26.776+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:27.385 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:27.383+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:27.751 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:27.749+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:28.344 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:28.342+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:28.734 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:28.732+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:29.326 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:29.324+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:29.730 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:29.728+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:30.336 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:30.334+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:30.764 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:30.762+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:31.367 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:31.365+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:31.720 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:31.718+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:32.329 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:32.327+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:32.752 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:32.750+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:33.337 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:33.335+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:33.764 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:33.762+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:34.293 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:34.291+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:34.811 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:34.809+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:35.331 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:35.329+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:35.784 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:35.782+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:36.290 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:36.288+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:36.794 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:36.792+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:37.306 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:37.304+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:37.747 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:37.745+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:38.273 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:38.271+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:38.782 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:38.780+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:39.317 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:39.315+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:39.771 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:39.769+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:40.346 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:40.344+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:40.784 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:40.782+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:41.373 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:41.371+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:41.775 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:41.773+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:42.404 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:42.402+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:42.735 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:42.733+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:43.392 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:43.389+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:43.729 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:43.727+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:44.355 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:44.353+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:44.722 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:44.720+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:45.338 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:45.336+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:45.706 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:45.704+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:46.348 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:46.345+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:46.747 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:46.745+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:47.350 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:47.348+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:47.706 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:47.704+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:48.395 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:48.393+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:48.739 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:48.737+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:49.389 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:49.387+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:49.696 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:49.694+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:50.384 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:50.381+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:50.739 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:50.737+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:51.396 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:51.394+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:51.767 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:51.765+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:52.375 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:52.373+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:52.730 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:52.728+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:53.341 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:53.339+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:53.748 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:53.746+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:54.391 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:54.389+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:54.729 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:54.727+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:55.399 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:55.396+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:55.761 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:55.759+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:56.402 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:56.400+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:56.748 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:56.746+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:57.385 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:57.383+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:57.724 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:57.722+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:58.420 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:58.418+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:58.730 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:58.727+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:59.405 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:18:59.403+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:18:59.688 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:18:59.686+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:00.447 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:00.445+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:00.672 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:00.669+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:01.412 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:01.410+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:01.637 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:01.635+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:02.416 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:02.414+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:02.638 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:02.636+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:03.410 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:03.409+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:03.642 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:03.641+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:04.420 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:04.419+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:04.610 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:04.609+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:05.446 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:05.445+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:05.579 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:05.578+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:06.470 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:06.469+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:06.589 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:06.588+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:07.517 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:07.515+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:07.623 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:07.622+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:08.535 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:08.535+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:08.607 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:08.606+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:09.552 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:09.551+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:09.561 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:09.560+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:10.532 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:10.531+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:10.582 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:10.581+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:11.502 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:11.501+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:11.629 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:11.628+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:12.503 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:12.502+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:12.584 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:12.583+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:13.510 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:13.509+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:13.582 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:13.581+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:14.542 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:14.541+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:14.623 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:14.622+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:15.540 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:15.538+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:15.577 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:15.576+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:16.518 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:16.517+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:16.561 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:16.560+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:17.490 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:17.489+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:17.546 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:17.545+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:18.508 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:18.507+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:18.517 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:18.516+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:19.480 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:19.478+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:19.499 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:19.498+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:20.482 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:20.481+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:20.513 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:20.512+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:21.464 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:21.463+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:21.539 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:21.538+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:22.465 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:22.464+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:22.578 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:22.577+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:23.492 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:23.490+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:23.573 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:23.571+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:24.473 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:24.471+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:24.617 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:24.615+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:25.428 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:25.426+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:25.575 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:25.573+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:26.447 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:26.445+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:26.568 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:26.566+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:27.445 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:27.443+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:27.619 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:27.617+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:28.417 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:28.414+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:28.659 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:28.657+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:29.449 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:29.447+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:29.658 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:29.656+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:30.415 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:30.413+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:30.658 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:30.656+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:31.369 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:31.367+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:31.682 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:31.680+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:32.332 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:32.330+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:32.670 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:32.668+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:33.331 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:33.329+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:33.699 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:33.697+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:34.289 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:34.287+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:34.710 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:34.708+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:35.246 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:35.244+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:35.694 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:35.692+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:36.285 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:36.283+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:36.688 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:36.686+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:37.262 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:37.260+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:37.716 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:37.714+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:38.300 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:38.298+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:38.676 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:38.674+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:39.286 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:39.284+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:39.713 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:39.711+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:40.321 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:40.319+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:40.729 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:40.727+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:41.338 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:41.336+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:41.710 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:41.708+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:42.336 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:42.333+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:42.664 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:42.662+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:43.304 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:43.302+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:43.708 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:43.706+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:44.270 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:44.268+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:44.741 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:44.739+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:45.298 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:45.296+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:45.722 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:45.720+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:46.286 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:46.284+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:46.754 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:46.752+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:47.324 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:47.322+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:47.715 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:47.713+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:48.328 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:48.325+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:48.740 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:48.738+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:49.322 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:49.320+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:49.727 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:49.725+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:50.370 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:50.368+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:50.759 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:50.757+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:51.394 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:51.392+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:51.750 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:51.748+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:52.353 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:52.351+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:52.785 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:52.783+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:53.310 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:53.308+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:53.758 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:53.756+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:54.291 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:54.289+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:54.766 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:54.764+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:55.261 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:55.259+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:55.732 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:55.730+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:56.303 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:56.301+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:56.728 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:56.726+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:57.318 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:57.316+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:57.699 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:57.697+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:58.281 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:58.279+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:58.716 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:58.714+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:59.264 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:19:59.262+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:19:59.728 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:19:59.725+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:00.244 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:00.242+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:00.725 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:00.724+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:01.260 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:01.259+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:01.694 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:01.693+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:02.218 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:02.217+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:02.670 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:02.668+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:03.200 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:03.199+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:03.644 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:03.642+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:04.168 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:04.167+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:04.674 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:04.673+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:05.156 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:05.155+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:05.674 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:05.673+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:06.145 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:06.144+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:06.714 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:06.713+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:07.189 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:07.188+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:07.729 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:07.728+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:08.168 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:08.167+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:08.738 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:08.737+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:09.199 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:09.198+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:09.741 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:09.740+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:10.218 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:10.216+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:10.777 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:10.775+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:11.239 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:11.238+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:11.788 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:11.787+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:12.287 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:12.286+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:12.775 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:12.774+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:13.248 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:13.247+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:13.785 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:13.784+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:14.240 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:14.238+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:14.762 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:14.761+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:15.196 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:15.195+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:15.777 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:15.776+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:16.154 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:16.153+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:16.782 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:16.781+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:17.189 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:17.188+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:17.779 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:17.778+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:18.172 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:18.171+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:18.731 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:18.730+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:19.134 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:19.133+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:19.779 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:19.778+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:20.126 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:20.126+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:20.801 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:20.800+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:21.127 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:21.125+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:21.767 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:21.765+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:22.095 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:22.093+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:22.780 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:22.778+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:23.049 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:23.047+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:23.817 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:23.815+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:24.010 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:24.008+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:24.840 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:24.838+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:24.965 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:24.963+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:25.861 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:25.859+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:25.984 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:25.982+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:26.868 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:26.866+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:26.937 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:26.935+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:27.908 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:27.906+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:27.972 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:27.970+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:28.920 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:28.918+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:28.981 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:28.979+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:29.945 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:29.943+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:29.947 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:29.945+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:30.899 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:30.897+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:30.945 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:30.944+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:31.904 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:31.902+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:31.954 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:31.952+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:32.885 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:32.883+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:32.951 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:32.949+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:33.911 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:33.909+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:33.978 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:33.974+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:34.866 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:34.864+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:34.958 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:34.956+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:35.830 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:35.828+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:36.001 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:35.998+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:36.786 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:36.784+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:37.008 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:37.006+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:37.807 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:37.805+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:37.963 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:37.962+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:38.821 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:38.818+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:38.950 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:38.948+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:39.795 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:39.793+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:39.976 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:39.974+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:40.780 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:40.778+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:40.936 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:40.934+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:41.823 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:41.820+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:41.889 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:41.887+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:42.813 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:42.811+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:42.852 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:42.850+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:43.828 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:43.826+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:43.885 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:43.883+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:44.822 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:44.820+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:44.919 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:44.917+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:45.853 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:45.851+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:45.878 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:45.876+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:46.842 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:46.840+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:46.899 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:46.897+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:47.836 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:47.834+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:47.943 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:47.941+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:48.804 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:48.801+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:48.904 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:48.902+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:49.754 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:49.752+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:49.896 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:49.893+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:50.787 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:50.785+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:50.861 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:50.859+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:51.826 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:51.824+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:51.828 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:51.826+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:52.812 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:52.810+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:52.874 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:52.872+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:53.780 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:53.778+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:53.919 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:53.917+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:54.755 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:54.753+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:54.960 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:54.958+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:55.712 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:55.710+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:55.986 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:55.984+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:56.676 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:56.674+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:56.953 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:56.951+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:57.650 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:57.649+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:57.975 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:57.973+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:58.634 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:58.632+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:58.963 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:58.962+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:59.630 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:20:59.629+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:20:59.978 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:20:59.976+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:00.665 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:00.664+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:00.996 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:00.995+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:01.698 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:01.697+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:02.038 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:02.037+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:02.729 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:02.728+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:03.078 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:03.077+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:03.706 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:03.705+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:04.110 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:04.109+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:04.735 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:04.734+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:05.114 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:05.113+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:05.730 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:05.729+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:06.106 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:06.105+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:06.699 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:06.698+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:07.125 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:07.124+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:07.680 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:07.679+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:08.121 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:08.120+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:08.655 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:08.654+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:09.124 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:09.123+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:09.672 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:09.671+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:10.138 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:10.137+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:10.695 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:10.694+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:11.114 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:11.113+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:11.727 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:11.726+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:12.068 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:12.067+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:12.689 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:12.688+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:13.042 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:13.041+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:13.697 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:13.696+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:14.047 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:14.046+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:14.655 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:14.654+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:15.061 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:15.060+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:15.609 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:15.608+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:16.076 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:16.075+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:16.614 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:16.613+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:17.032 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:17.031+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:17.645 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:17.644+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:18.040 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:18.039+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:18.669 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:18.667+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:19.061 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:19.058+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:19.687 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:19.685+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:20.067 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:20.065+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:20.640 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:20.638+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:21.096 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:21.094+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:21.621 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:21.619+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:22.051 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:22.049+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:22.629 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:22.627+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:23.003 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:23.001+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:23.658 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:23.656+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:23.982 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:23.980+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:24.641 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:24.639+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:24.966 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:24.964+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:25.663 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:25.661+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:25.972 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:25.970+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:26.708 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:26.706+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:26.982 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:26.980+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:27.706 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:27.703+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:27.985 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:27.983+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:28.713 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:28.710+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:28.938 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:28.936+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:29.738 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:29.736+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:29.954 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:29.952+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:30.693 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:30.691+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:30.971 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:30.969+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:31.710 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:31.708+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:31.979 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:31.977+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:32.709 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:32.707+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:33.023 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:33.021+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:33.759 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:33.757+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:34.027 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:34.025+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:34.778 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:34.776+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:35.018 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:35.016+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:35.765 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:35.763+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:36.035 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:36.033+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:36.798 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:36.796+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:37.025 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:37.023+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:37.803 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:37.801+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:38.061 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:38.059+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:38.827 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:38.825+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:39.088 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:39.086+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:39.785 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:39.783+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:40.121 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:40.119+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:40.736 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:40.734+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:41.148 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:41.146+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:41.738 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:41.736+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:42.157 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:42.155+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:42.762 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:42.760+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:43.196 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:43.194+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:43.746 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:43.744+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:44.208 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:44.206+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:44.766 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:44.764+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:45.258 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:45.255+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:45.808 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:45.806+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:46.242 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:46.240+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:46.827 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:46.825+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:47.200 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:47.198+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:47.804 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:47.802+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:48.216 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:48.214+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:48.789 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:48.786+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:49.258 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:49.256+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:49.774 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:49.771+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:50.266 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:50.264+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:50.741 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:50.738+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:51.306 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:51.304+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:51.707 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:51.705+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:52.292 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:52.290+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:52.756 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:52.754+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:53.306 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:53.304+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:53.785 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:53.783+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:54.316 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:54.314+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:54.820 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:54.818+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:55.297 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:55.295+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:55.855 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:55.854+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:56.278 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:56.276+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:56.845 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:56.844+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:57.287 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:57.286+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:57.840 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:57.839+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:58.303 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:58.302+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:58.832 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:58.831+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:59.292 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:21:59.291+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:21:59.807 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:21:59.806+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:00.259 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:00.258+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:00.833 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:00.832+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:01.271 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:01.270+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:01.835 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:01.834+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:02.296 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:02.295+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:02.876 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:02.875+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:03.265 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:03.264+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:03.842 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:03.841+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:04.222 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:04.220+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:04.836 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:04.835+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:05.262 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:05.261+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:05.801 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:05.800+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:06.215 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:06.214+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:06.784 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:06.783+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:07.261 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:07.260+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:07.758 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:07.757+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:08.257 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:08.256+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:08.720 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:08.719+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:09.302 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:09.301+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:09.764 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:09.763+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:10.296 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:10.295+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:10.734 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:10.733+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:11.296 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:11.295+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:11.736 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:11.735+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:12.268 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:12.267+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:12.767 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:12.766+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:13.276 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:13.275+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:13.746 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:13.745+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:14.309 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:14.308+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:14.766 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:14.765+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:15.302 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:15.301+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:15.805 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:15.804+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:16.323 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:16.322+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:16.773 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:16.772+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:17.284 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:17.282+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:17.731 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:17.730+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:18.303 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:18.301+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:18.734 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:18.732+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:19.257 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:19.255+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:19.686 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:19.684+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:20.233 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:20.231+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:20.729 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:20.727+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:21.235 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:21.233+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:21.701 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:21.699+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:22.280 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:22.278+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:22.723 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:22.721+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:23.272 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:23.270+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:23.688 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:23.686+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:24.237 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:24.234+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:24.644 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:24.642+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:25.232 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:25.230+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:25.632 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:25.630+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:26.199 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:26.197+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:26.619 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:26.617+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:27.186 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:27.184+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:27.589 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:27.587+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:28.138 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:28.136+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:28.592 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:28.590+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:29.175 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:29.173+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:29.623 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:29.621+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:30.188 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:30.186+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:30.662 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:30.660+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:31.144 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:31.142+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:31.696 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:31.694+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:32.164 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:32.162+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:32.746 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:32.740+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:33.120 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:33.118+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:33.713 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:33.711+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:34.156 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:34.154+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:34.671 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:34.669+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:35.162 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:35.160+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:35.655 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:35.653+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:36.167 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:36.165+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:36.655 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:36.653+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:37.209 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:37.207+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:37.658 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:37.656+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:38.183 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:38.181+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:38.700 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:38.698+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:39.175 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:39.173+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:39.683 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:39.681+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:40.146 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:40.144+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:40.716 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:40.714+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:41.166 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:41.164+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:41.738 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:41.736+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:42.179 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:42.177+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:42.714 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:42.712+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:43.216 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:43.214+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:43.706 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:43.704+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:44.179 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:44.177+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:44.674 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:44.672+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:45.215 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:45.213+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:45.659 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:45.657+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:46.179 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:46.177+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:46.654 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:46.652+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:47.190 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:47.188+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:47.651 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:47.650+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:48.218 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:48.216+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:48.638 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:48.636+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:49.253 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:49.251+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:49.623 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:49.621+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:50.298 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:50.296+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:50.588 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:50.586+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:51.302 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:51.300+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:51.546 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:51.544+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:52.276 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:52.274+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:52.574 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:52.572+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:53.277 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:53.275+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:53.556 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:53.554+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:54.322 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:54.320+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:54.606 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:54.604+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:55.365 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:55.364+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:55.646 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:55.645+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:56.366 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:56.365+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:56.681 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:56.681+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:57.326 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:57.325+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:57.657 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:57.656+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:58.343 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:58.342+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:58.645 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:58.644+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:59.362 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:22:59.361+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:22:59.656 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:22:59.655+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:00.370 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:00.369+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:00.631 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:00.630+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:01.331 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:01.330+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:01.606 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:01.605+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:02.295 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:02.294+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:02.623 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:02.622+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:03.333 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:03.332+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:03.657 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:03.656+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:04.291 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:04.290+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:04.696 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:04.695+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:05.247 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:05.246+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:05.657 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:05.656+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:06.296 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:06.295+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:06.638 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:06.637+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:07.323 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:07.322+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:07.635 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:07.635+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:08.287 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:08.286+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:08.649 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:08.648+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:09.282 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:09.281+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:09.689 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:09.688+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:10.287 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:10.285+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:10.701 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:10.700+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:11.257 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:11.256+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:11.682 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:11.681+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:12.211 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:12.210+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:12.731 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:12.730+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:13.174 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:13.173+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:13.744 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:13.743+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:14.151 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:14.150+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:14.776 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:14.776+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:15.133 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:15.130+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:15.734 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:15.732+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:16.120 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:16.118+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:16.728 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:16.726+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:17.107 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:17.105+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:17.759 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:17.757+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:18.132 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:18.130+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:18.809 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:18.807+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:19.096 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:19.094+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:19.780 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:19.778+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:20.075 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:20.073+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:20.785 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:20.783+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:21.081 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:21.079+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:21.799 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:21.797+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:22.058 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:22.057+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:22.771 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:22.769+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:23.087 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:23.085+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:23.751 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:23.749+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:24.099 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:24.097+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:24.713 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:24.711+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:25.075 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:25.073+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:25.679 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:25.677+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:26.117 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:26.115+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:26.648 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:26.646+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:27.122 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:27.120+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:27.629 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:27.627+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:28.140 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:28.138+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:28.670 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:28.669+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:29.103 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:29.101+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:29.718 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:29.716+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:30.115 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:30.113+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:30.670 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:30.668+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:31.087 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:31.085+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:31.676 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:31.673+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:32.072 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:32.070+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:32.717 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:32.715+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:33.086 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:33.084+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:33.692 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:33.689+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:34.095 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:34.093+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:34.719 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:34.717+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:35.081 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:35.079+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:35.729 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:35.725+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:36.120 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:36.118+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:36.773 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:36.771+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:37.157 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:37.155+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:37.743 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:37.741+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:38.177 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:38.175+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:38.747 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:38.745+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:39.181 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:39.180+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:39.743 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:39.741+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:40.162 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:40.160+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:40.765 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:40.764+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:41.143 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:41.141+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:41.811 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:41.809+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:42.157 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:42.155+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:42.822 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:42.820+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:43.146 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:43.144+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:43.807 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:43.805+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:44.158 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:44.156+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:44.845 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:44.843+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:45.197 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:45.195+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:45.890 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:45.888+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:46.199 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:46.197+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:46.921 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:46.919+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:47.167 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:47.165+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:47.972 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:47.970+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:48.150 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:48.148+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:48.952 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:48.950+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:49.188 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:49.186+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:49.916 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:49.914+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:50.231 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:50.229+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:50.961 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:50.959+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:51.241 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:51.239+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:51.971 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:51.970+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:52.239 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:52.237+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:52.948 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:52.948+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:53.249 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:53.248+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:53.901 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:53.900+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:54.222 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:54.222+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:54.908 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:54.907+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:55.220 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:55.219+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:55.950 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:55.949+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:56.176 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:56.174+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:56.932 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:56.931+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:57.203 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:57.202+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:57.899 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:57.898+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:58.168 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:58.166+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:58.945 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:58.944+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:59.149 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:23:59.148+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:23:59.970 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:23:59.969+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:00.167 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:00.166+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:00.988 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:00.987+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:01.170 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:01.169+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:01.949 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:01.948+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:02.174 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:02.173+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:02.970 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:02.969+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:03.200 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:03.199+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:03.972 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:03.971+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:04.157 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:04.155+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:05.001 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:05.000+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:05.130 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:05.129+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:06.034 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:06.034+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:06.087 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:06.086+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:07.019 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:07.018+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:07.060 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:07.059+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:08.025 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:08.024+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:08.031 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:08.030+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:09.006 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:09.005+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:09.056 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:09.055+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:10.026 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:10.025+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:10.054 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:10.053+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:11.029 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:11.028+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:11.100 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:11.099+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:12.043 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:12.042+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:12.055 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:12.054+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:13.017 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:13.014+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:13.057 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:13.055+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:13.979 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:13.977+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:14.026 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:14.024+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:14.982 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:14.980+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:14.993 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:14.991+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:16.008 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:16.006+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:16.010 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:16.008+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:16.960 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:16.958+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:16.994 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:16.992+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:17.945 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:17.943+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:18.034 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:18.032+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:18.924 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:18.922+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:19.060 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:19.058+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:19.908 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:19.906+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:20.106 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:20.104+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:20.859 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:20.857+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:21.096 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:21.094+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:21.813 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:21.811+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:22.103 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:22.101+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:22.828 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:22.826+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:23.150 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:23.148+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:23.856 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:23.854+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:24.105 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:24.102+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:24.854 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:24.852+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:25.108 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:25.106+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:25.819 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:25.817+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:26.102 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:26.100+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:26.806 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:26.804+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:27.146 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:27.144+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:27.769 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:27.767+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:28.140 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:28.138+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:28.724 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:28.722+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:29.151 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:29.149+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:29.773 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:29.771+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:30.188 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:30.186+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:30.807 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:30.805+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:31.219 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:31.217+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:31.809 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:31.807+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:32.199 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:32.197+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:32.839 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:32.838+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:33.223 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:33.221+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:33.857 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:33.855+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:34.240 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:34.238+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:34.876 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:34.874+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:35.227 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:35.225+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:35.902 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:35.900+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:36.261 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:36.258+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:36.900 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:36.898+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:37.301 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:37.299+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:37.872 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:37.870+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:38.257 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:38.255+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:38.884 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:38.882+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:39.292 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:39.290+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:39.886 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:39.884+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:40.273 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:40.271+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:40.920 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:40.918+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:41.281 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:41.279+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:41.897 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:41.895+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:42.307 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:42.305+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:42.876 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:42.874+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:43.278 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:43.276+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:43.867 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:43.865+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:44.328 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:44.326+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:44.881 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:44.879+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:45.348 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:45.346+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:45.904 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:45.902+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:46.348 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:46.346+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:46.879 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:46.877+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:47.366 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:47.364+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:47.876 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:47.874+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:48.374 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:48.372+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:48.854 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:48.852+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:49.404 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:49.402+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:49.860 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:49.858+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:50.451 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:50.450+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:50.849 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:50.848+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:51.471 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:51.470+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:51.815 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:51.814+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:52.438 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:52.437+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:52.811 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:52.810+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:53.445 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:53.444+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:53.856 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:53.855+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:54.465 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:54.464+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:54.814 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:54.813+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:55.506 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:55.505+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:55.839 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:55.838+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:56.479 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:56.478+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:56.876 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:56.875+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:57.458 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:57.457+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:57.836 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:57.835+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:58.473 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:58.472+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:58.793 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:58.792+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:59.433 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:24:59.432+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:24:59.789 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:24:59.788+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:00.402 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:00.401+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:00.740 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:00.739+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:01.400 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:01.399+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:01.766 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:01.765+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:02.407 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:02.406+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:02.757 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:02.756+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:03.431 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:03.430+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:03.733 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:03.732+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:04.390 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:04.389+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:04.705 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:04.704+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:05.406 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:05.406+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:05.668 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:05.667+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:06.368 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:06.367+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:06.705 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:06.704+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:07.400 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:07.399+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:07.710 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:07.709+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:08.443 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:08.442+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:08.756 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:08.755+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:09.436 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:09.435+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:09.800 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:09.799+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:10.444 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:10.443+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:10.759 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:10.757+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:11.481 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:11.479+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:11.750 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:11.748+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:12.491 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:12.490+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:12.781 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:12.779+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:13.525 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:13.523+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:13.750 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:13.748+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:14.499 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:14.497+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:14.776 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:14.774+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:15.497 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:15.495+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:15.741 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:15.739+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:16.515 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:16.513+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:16.786 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:16.784+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:17.472 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:17.470+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:17.754 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:17.752+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:18.491 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:18.489+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:18.772 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:18.770+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:19.464 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:19.462+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:19.822 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:19.820+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:20.415 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:20.413+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:20.780 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:20.778+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:21.422 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:21.420+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:21.736 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:21.734+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:22.457 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:22.455+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:22.740 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:22.738+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:23.441 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:23.439+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:23.744 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:23.742+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:24.439 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:24.438+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:24.779 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:24.777+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:25.398 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:25.395+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:25.799 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:25.797+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:26.438 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:26.436+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:26.836 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:26.834+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:27.427 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:27.425+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:27.882 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:27.880+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:28.405 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:28.403+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:28.849 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:28.844+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:29.424 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:29.422+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:29.870 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:29.868+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:30.426 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:30.424+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:30.849 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:30.847+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:31.468 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:31.466+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:31.826 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:31.824+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:32.439 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:32.437+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:32.814 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:32.812+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:33.407 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:33.405+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:33.810 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:33.808+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:34.390 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:34.388+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:34.791 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:34.789+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:35.424 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:35.422+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:35.749 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:35.747+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:36.402 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:36.400+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:36.768 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:36.766+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:37.357 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:37.355+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:37.720 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:37.718+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:38.336 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:38.334+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:38.742 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:38.740+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:39.314 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:39.312+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:39.715 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:39.713+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:40.335 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:40.333+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:40.707 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:40.705+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:41.323 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:41.322+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:41.684 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:41.682+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:42.328 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:42.326+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:42.652 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:42.650+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:43.320 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:43.318+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:43.700 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:43.698+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:44.327 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:44.325+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:44.718 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:44.716+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:45.332 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:45.330+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:45.689 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:45.687+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:46.316 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:46.314+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:46.660 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:46.658+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:47.274 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:47.272+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:47.669 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:47.667+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:48.285 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:48.283+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:48.655 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:48.654+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:49.238 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:49.237+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:49.642 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:49.641+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:50.282 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:50.281+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:50.598 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:50.597+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:51.242 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:51.241+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:51.604 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:51.603+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:52.238 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:52.237+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:52.576 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:52.575+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:53.237 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:53.236+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:53.570 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:53.569+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:54.231 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:54.230+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:54.583 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:54.582+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:55.198 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:55.196+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:55.617 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:55.616+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:56.232 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:56.231+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:56.620 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:56.619+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:57.280 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:57.279+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:57.669 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:57.668+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:58.283 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:58.282+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:58.681 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:58.680+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:59.240 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:25:59.238+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:25:59.654 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:25:59.653+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:00.244 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:00.243+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:00.684 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:00.683+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:01.240 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:01.239+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:01.684 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:01.683+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:02.280 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:02.279+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:02.699 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:02.698+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:03.313 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:03.312+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:03.688 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:03.687+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:04.327 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:04.326+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:04.735 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:04.734+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:05.279 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:05.277+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:05.700 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:05.699+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:06.282 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:06.281+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:06.719 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:06.717+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:07.249 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:07.249+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:07.731 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:07.730+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:08.259 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:08.259+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:08.771 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:08.771+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:09.244 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:09.242+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:09.744 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:09.742+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:10.280 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:10.278+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:10.757 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:10.755+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:11.310 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:11.308+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:11.789 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:11.787+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:12.297 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:12.296+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:12.788 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:12.786+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:13.297 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:13.295+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:13.818 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:13.816+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:14.281 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:14.280+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:14.842 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:14.840+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:15.260 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:15.258+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:15.862 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:15.860+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:16.307 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:16.305+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:16.848 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:16.846+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:17.351 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:17.349+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:17.840 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:17.838+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:18.322 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:18.320+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:18.852 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:18.850+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:19.338 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:19.336+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:19.812 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:19.810+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:20.322 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:20.320+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:20.859 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:20.857+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:21.274 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:21.272+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:21.816 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:21.814+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:22.308 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:22.306+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:22.774 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:22.772+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:23.351 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:23.349+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:23.726 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:23.724+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:24.333 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:24.331+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:24.692 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:24.690+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:25.362 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:25.360+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:25.731 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:25.729+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:26.353 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:26.351+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:26.696 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:26.695+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:27.370 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:27.368+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:27.721 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:27.719+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:28.374 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:28.373+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:28.718 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T22:26:28.716+0000 7f994396e700 0 -- 172.21.15.167:0/1156974075 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x55b130ed3000 msgr2=0x7f9928036570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-05T22:26:28.721 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:28.719+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:29.333 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:29.331+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:29.734 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:29.731+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:30.309 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:30.306+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:30.740 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:30.738+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:31.346 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:31.344+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:31.766 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:31.764+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:32.364 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:32.362+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:32.740 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:32.738+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:33.406 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:33.404+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:33.722 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:33.720+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:34.408 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:34.407+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:34.746 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:34.744+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:35.362 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:35.360+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:35.746 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:35.744+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:36.319 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:36.317+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:36.696 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:36.694+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:37.293 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:37.291+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:37.734 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:37.732+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:38.297 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:38.295+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:38.687 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:38.685+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:39.322 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:39.320+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:39.653 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:39.651+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:40.328 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:40.326+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:40.630 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:40.628+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:41.346 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:41.344+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:41.627 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:41.625+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:42.312 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:42.310+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:42.671 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:42.669+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:43.297 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:43.295+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:43.711 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:43.709+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:44.264 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:44.262+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:44.715 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:44.713+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:45.279 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:45.277+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:45.683 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:45.681+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:46.300 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:46.298+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:46.660 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:46.658+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:47.322 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:47.321+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:47.626 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:47.625+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:48.301 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:48.300+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:48.657 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:48.656+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:49.289 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:49.287+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:49.614 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:49.613+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:50.313 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:50.312+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:50.649 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:50.648+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:51.303 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:51.302+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:51.685 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:51.684+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:52.338 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:52.337+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:52.703 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:52.702+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:53.310 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:53.309+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:53.707 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:53.706+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:54.284 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:54.283+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:54.745 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:54.744+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:55.311 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:55.310+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:55.752 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:55.752+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:56.349 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:56.348+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:56.749 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:56.748+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:57.399 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:57.398+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:57.757 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:57.756+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:58.413 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:58.412+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:58.730 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:58.729+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:59.367 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:26:59.366+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:26:59.711 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:26:59.710+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:00.395 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:00.394+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:00.663 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:00.662+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:01.439 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:01.438+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:01.669 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:01.668+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:02.469 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:02.468+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:02.716 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:02.716+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:03.485 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:03.484+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:03.761 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:03.760+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:04.486 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:04.485+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:04.777 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:04.776+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:05.448 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:05.447+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:05.792 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:05.791+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:06.491 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:06.490+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:06.816 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:06.815+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:07.446 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:07.444+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:07.854 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:07.852+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:08.416 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:08.414+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:08.827 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:08.825+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:09.394 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:09.392+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:09.863 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:09.861+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:10.416 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:10.414+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:10.878 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:10.876+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:11.441 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:11.439+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:11.919 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:11.917+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:12.447 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:12.445+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:12.956 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:12.954+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:13.490 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:13.489+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:13.973 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:13.971+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:14.442 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:14.440+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:14.994 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:14.992+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:15.444 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:15.442+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:15.979 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:15.977+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:16.485 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:16.483+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:16.978 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:16.976+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:17.498 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:17.496+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:18.005 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:18.003+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:18.469 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:18.467+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:18.978 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:18.976+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:19.512 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:19.510+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:19.972 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:19.970+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:20.556 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:20.554+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:20.966 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:20.965+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:21.593 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:21.591+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:21.940 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:21.938+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:22.620 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:22.618+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:22.920 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:22.919+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:23.617 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:23.615+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:23.962 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:23.960+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:24.640 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:24.638+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:24.981 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:24.979+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:25.610 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:25.608+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:26.028 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:26.026+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:26.590 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:26.587+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:27.061 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:27.059+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:27.589 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:27.587+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:28.041 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:28.039+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:28.637 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:28.635+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:28.999 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:28.997+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:29.670 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:29.668+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:29.995 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:29.993+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:30.709 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:30.707+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:30.980 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:30.978+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:31.756 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:31.754+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:32.016 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:32.014+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:32.770 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:32.769+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:33.062 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:33.060+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:33.760 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:33.758+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:34.029 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:34.027+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:34.806 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:34.804+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:34.987 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:34.985+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:35.800 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:35.798+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:35.997 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:35.993+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:36.785 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:36.783+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:37.041 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:37.039+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:37.787 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:37.785+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:38.049 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:38.047+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:38.788 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:38.786+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:39.054 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:39.052+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:39.839 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:39.837+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:40.039 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:40.036+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:40.873 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:40.871+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:41.019 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:41.017+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:41.839 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:41.837+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:42.013 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:42.011+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:42.836 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:42.835+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:42.994 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:42.993+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:43.791 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:43.789+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:43.984 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:43.982+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:44.808 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:44.806+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:44.964 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:44.963+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:45.824 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:45.823+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:45.939 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:45.938+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:46.776 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:46.775+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:46.910 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:46.909+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:47.813 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:47.812+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:47.869 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:47.868+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:48.825 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:48.824+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:48.885 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:48.884+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:49.802 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:49.801+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:49.911 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:49.910+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:50.826 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:50.825+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:50.899 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:50.898+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:51.821 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:51.820+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:51.908 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:51.907+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:52.780 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:52.779+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:52.867 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:52.866+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:53.735 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:53.734+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:53.894 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:53.893+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:54.719 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:54.718+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:54.864 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:54.863+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:55.710 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:55.710+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:55.897 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:55.896+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:56.669 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:56.668+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:56.931 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:56.930+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:57.678 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:57.677+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:57.956 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:57.955+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:58.663 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:58.662+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:58.994 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:58.993+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:59.646 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:27:59.645+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:27:59.998 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:27:59.997+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:00.683 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:00.682+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:00.996 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:00.996+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:01.638 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:01.637+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:01.973 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:01.972+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:02.594 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:02.594+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:02.930 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:02.929+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:03.601 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:03.600+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:03.912 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:03.912+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:04.590 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:04.589+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:04.911 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:04.910+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:05.562 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:05.561+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:05.911 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:05.909+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:06.573 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:06.571+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:06.909 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:06.905+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:07.551 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:07.549+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:07.903 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:07.901+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:08.571 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:08.569+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:08.920 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:08.918+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:09.605 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:09.603+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:09.947 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:09.945+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:10.624 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:10.622+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:10.912 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:10.910+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:11.671 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:11.669+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:11.931 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:11.929+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:12.681 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:12.679+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:12.923 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:12.921+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:13.728 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:13.726+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:13.921 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:13.919+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:14.778 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:14.776+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:14.899 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:14.897+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:15.786 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:15.784+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:15.914 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:15.912+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:16.836 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:16.834+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:16.884 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:16.882+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:17.849 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:17.847+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:17.854 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:17.853+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:18.833 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:18.831+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:18.845 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:18.843+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:19.787 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:19.785+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:19.818 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:19.816+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:20.775 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:20.773+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:20.827 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:20.825+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:21.786 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:21.784+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:21.822 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:21.820+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:22.748 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:22.746+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:22.847 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:22.845+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:23.756 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:23.754+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:23.865 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:23.863+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:24.748 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:24.746+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:24.906 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:24.905+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:25.701 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:25.699+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:25.946 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:25.944+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:26.701 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:26.698+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:26.966 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:26.965+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:27.723 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:27.721+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:28.002 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:28.000+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:28.696 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:28.695+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:29.011 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:29.009+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:29.730 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:29.729+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:30.027 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:30.025+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:30.688 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:30.686+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:31.049 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:31.046+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:31.661 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:31.659+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:32.028 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:32.026+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:32.681 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:32.679+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:33.051 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:33.049+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:33.632 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:33.630+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:34.019 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:34.017+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:34.591 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:34.589+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:35.028 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:35.026+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:35.608 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:35.606+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:35.980 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:35.978+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:36.641 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:36.639+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:36.945 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:36.943+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:37.610 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:37.608+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:37.906 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:37.905+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:38.580 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:38.579+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:38.939 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:38.937+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:39.547 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:39.545+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:39.898 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:39.896+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:40.583 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:40.581+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:40.876 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:40.874+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:41.581 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:41.579+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:41.896 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:41.894+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:42.550 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:42.548+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:42.915 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:42.913+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:43.560 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:43.560+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:43.903 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:43.902+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:44.530 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:44.529+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:44.879 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:44.878+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:45.497 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:45.496+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:45.865 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:45.864+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:46.450 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:46.449+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:46.891 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:46.890+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:47.458 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:47.457+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:47.902 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:47.902+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:48.471 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:48.470+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:48.898 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:48.897+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:49.489 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:49.488+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:49.891 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:49.891+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:50.457 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:50.456+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:50.879 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:50.878+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:51.446 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:51.445+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:51.864 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:51.863+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:52.486 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:52.485+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:52.843 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:52.842+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:53.474 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:53.474+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:53.847 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:53.846+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:54.463 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:54.462+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:54.843 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:54.842+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:55.417 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:55.416+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:55.798 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:55.797+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:56.429 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:56.428+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:56.808 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:56.808+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:57.452 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:57.452+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:57.765 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:57.764+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:58.468 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:58.467+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:58.783 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:58.782+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:59.433 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:28:59.432+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:28:59.770 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:28:59.769+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:00.482 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:00.482+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:00.755 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:00.754+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:01.501 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:01.500+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:01.757 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:01.757+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:02.481 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:02.480+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:02.792 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:02.792+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:03.499 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:03.497+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:03.770 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:03.768+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:04.550 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:04.548+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:04.801 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:04.799+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:05.510 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:05.508+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:05.800 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:05.798+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:06.557 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:06.555+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:06.771 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:06.769+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:07.597 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:07.595+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:07.730 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:07.728+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:08.572 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:08.570+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:08.695 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:08.693+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:09.540 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:09.538+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:09.673 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:09.671+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:10.572 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:10.570+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:10.656 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:10.654+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:11.603 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:11.601+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:11.650 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:11.648+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:12.626 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:12.625+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:12.656 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:12.654+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:13.614 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:13.613+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:13.667 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:13.665+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:14.613 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:14.611+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:14.624 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:14.622+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:15.569 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:15.567+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:15.621 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:15.619+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:16.563 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:16.561+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:16.590 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:16.588+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:17.566 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:17.565+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:17.602 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:17.601+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:18.529 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:18.527+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:18.645 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:18.643+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:19.506 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:19.504+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:19.666 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:19.665+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:20.532 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:20.530+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:20.690 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:20.689+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:21.510 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:21.508+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:21.657 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:21.655+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:22.486 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:22.484+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:22.649 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:22.647+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:23.485 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:23.483+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:23.613 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:23.611+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:24.453 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:24.451+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:24.637 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:24.635+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:25.417 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:25.415+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:25.653 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:25.651+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:26.447 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:26.445+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:26.653 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:26.649+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:27.453 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:27.451+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:27.689 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:27.687+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:28.443 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:28.441+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:28.711 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:28.710+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:29.395 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:29.393+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:29.698 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:29.697+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:30.391 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:30.389+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:30.680 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:30.678+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:31.350 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:31.348+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:31.651 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:31.649+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:32.364 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:32.362+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:32.668 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:32.666+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:33.368 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:33.366+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:33.632 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:33.630+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:34.389 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:34.387+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:34.682 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:34.680+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:35.426 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:35.425+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:35.667 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:35.665+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:36.404 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:36.403+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:36.707 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:36.705+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:37.449 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:37.447+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:37.664 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:37.663+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:38.471 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:38.469+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:38.634 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:38.632+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:39.490 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:39.489+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:39.630 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:39.628+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:40.448 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:40.447+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:40.582 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:40.580+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:41.419 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:41.418+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:41.624 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:41.623+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:42.415 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:42.414+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:42.638 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:42.637+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:43.412 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:43.411+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:43.648 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:43.648+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:44.379 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:44.378+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:44.697 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:44.696+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:45.371 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:45.370+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:45.696 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:45.695+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:46.390 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:46.390+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:46.722 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:46.721+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:47.358 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:47.357+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:47.684 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:47.684+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:48.377 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:48.376+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:48.715 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:48.714+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:49.338 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:49.337+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:49.715 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:49.714+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:50.332 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:50.332+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:50.734 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:50.734+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:51.350 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:51.350+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:51.760 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:51.759+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:52.351 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:52.350+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:52.784 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:52.783+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:53.335 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:53.334+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:53.834 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:53.833+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:54.358 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:54.357+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:54.796 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:54.795+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:55.370 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:55.369+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:55.799 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:55.798+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:56.406 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:56.405+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:56.779 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:56.778+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:57.402 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:57.401+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:57.753 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:57.753+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:58.404 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:58.403+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:58.731 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:58.730+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:59.358 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:29:59.357+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:29:59.741 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:29:59.741+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:00.334 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:00.333+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:00.746 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:00.745+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:01.377 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:01.376+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:01.721 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:01.719+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:02.374 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:02.372+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:02.707 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:02.705+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:03.411 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:03.409+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:03.733 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:03.731+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:04.386 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:04.384+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:04.769 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:04.767+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:05.365 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:05.363+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:05.746 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:05.744+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:06.403 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:06.401+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:06.771 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:06.769+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:07.405 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:07.404+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:07.727 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:07.725+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:08.389 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:08.387+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:08.685 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:08.683+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:09.396 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:09.394+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:09.657 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:09.655+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:10.354 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:10.352+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:10.672 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:10.670+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:11.345 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:11.343+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:11.719 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:11.717+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:12.395 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:12.393+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:12.691 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:12.689+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:13.425 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:13.423+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:13.701 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:13.699+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:14.439 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:14.437+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:14.682 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:14.680+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:15.469 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:15.467+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:15.717 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:15.715+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:16.468 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:16.467+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:16.705 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:16.703+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:17.514 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:17.512+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:17.756 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:17.753+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:18.517 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:18.515+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:18.736 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:18.734+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:19.469 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:19.467+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:19.706 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:19.704+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:20.423 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:20.421+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:20.669 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:20.667+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:21.426 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:21.424+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:21.677 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:21.675+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:22.449 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:22.447+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:22.686 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:22.684+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:23.485 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:23.483+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:23.660 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:23.658+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:24.583 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:24.525+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:24.686 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:24.684+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:25.552 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:25.550+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:25.714 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:25.712+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:26.503 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:26.501+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:26.687 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:26.685+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:27.474 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:27.472+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:27.712 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:27.710+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:28.495 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:28.493+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:28.738 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:28.737+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:29.462 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:29.460+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:29.785 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:29.783+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:30.513 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:30.511+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:30.806 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:30.805+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:31.487 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:31.485+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:31.835 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:31.833+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:32.447 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:32.445+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:32.854 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:32.852+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:33.452 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:33.450+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:33.898 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:33.897+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:34.487 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:34.485+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:34.907 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:34.905+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:35.521 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:35.519+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:35.876 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:35.874+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:36.535 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:36.533+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:36.905 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:36.903+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:37.553 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:37.552+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:37.929 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:37.927+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:38.529 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:38.527+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:38.909 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:38.907+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:39.554 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:39.551+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:39.925 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:39.924+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:40.575 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:40.574+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:40.891 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:40.890+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:41.605 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:41.604+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:41.938 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:41.937+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:42.591 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:42.590+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:42.913 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:42.912+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:43.574 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:43.573+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:43.944 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:43.943+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:44.551 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:44.550+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:44.983 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:44.982+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:45.565 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:45.564+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:45.951 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:45.950+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:46.578 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:46.577+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:46.985 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:46.984+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:47.543 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:47.542+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:47.974 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:47.973+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:48.553 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:48.550+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:49.015 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:49.014+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:49.539 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:49.538+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:50.063 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:50.062+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:50.516 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:50.515+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:51.018 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:51.016+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:51.488 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:51.487+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:52.010 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:52.010+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:52.452 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:52.451+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:53.002 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:53.001+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:53.468 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:53.467+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:53.968 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:53.967+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:54.435 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:54.434+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:54.934 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:54.933+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:55.394 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:55.393+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:55.886 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:55.885+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:56.386 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:56.385+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:56.905 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:56.904+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:57.432 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:57.431+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:57.955 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:57.953+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:58.459 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:58.458+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:59.004 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:30:59.003+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:30:59.472 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:30:59.471+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:00.031 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:00.029+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:00.423 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:00.421+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:01.035 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:01.033+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:01.410 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:01.408+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:02.065 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:02.063+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:02.451 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:02.449+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:03.067 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:03.065+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:03.481 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:03.479+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:04.077 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:04.075+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:04.508 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:04.506+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:05.039 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:05.037+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:05.494 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:05.492+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:06.025 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:06.023+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:06.464 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:06.462+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:07.063 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:07.061+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:07.465 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:07.464+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:08.085 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:08.083+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:08.457 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:08.455+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:09.100 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:09.098+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:09.506 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:09.504+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:10.074 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:10.072+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:10.469 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:10.467+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:11.103 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:11.101+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:11.432 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:11.430+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:12.089 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:12.087+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:12.480 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:12.478+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:13.104 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:13.103+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:13.499 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:13.497+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:14.152 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:14.151+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:14.527 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:14.525+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:15.108 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:15.106+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:15.516 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:15.514+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:16.156 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:16.154+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:16.467 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:16.466+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:17.154 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:17.152+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:17.518 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:17.516+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:18.180 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:18.178+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:18.485 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:18.483+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:19.193 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:19.191+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:19.495 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:19.493+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:20.204 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:20.203+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:20.483 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:20.481+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:21.243 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:21.241+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:21.444 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:21.442+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:22.288 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:22.286+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:22.487 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:22.485+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:23.316 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:23.314+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:23.519 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:23.517+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:24.287 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:24.285+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:24.567 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:24.565+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:25.247 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:25.245+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:25.591 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:25.589+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:26.207 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:26.206+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:26.569 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:26.567+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:27.214 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:27.213+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:27.671 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:27.594+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:28.221 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:28.219+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:28.616 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:28.614+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:29.202 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:29.200+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:29.580 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:29.578+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:30.192 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:30.191+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:30.562 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:30.560+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:31.144 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:31.142+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:31.598 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:31.596+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:32.145 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:32.143+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:32.634 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:32.633+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:33.110 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:33.109+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:33.650 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:33.648+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:34.082 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:34.080+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:34.693 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:34.691+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:35.102 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:35.101+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:35.643 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:35.641+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:36.060 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:36.059+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:36.617 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:36.615+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:37.031 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:37.029+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:37.633 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:37.632+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:38.025 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:38.024+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:38.649 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:38.649+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:38.994 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:38.993+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:39.605 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:39.604+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:40.045 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:40.043+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:40.599 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:40.598+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:41.031 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:41.030+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:41.606 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:41.605+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:42.035 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:42.034+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:42.623 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:42.622+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:43.054 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:43.053+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:43.590 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:43.589+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:44.078 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:44.077+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:44.584 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:44.584+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:45.086 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:45.086+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:45.569 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:45.568+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:46.122 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:46.121+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:46.547 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:46.546+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:47.097 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:47.096+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:47.583 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:47.583+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:48.131 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:48.130+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:48.563 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:48.562+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:49.163 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:49.162+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:49.526 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:49.525+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:50.120 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:50.119+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:50.562 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:50.561+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:51.086 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:51.085+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:51.598 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:51.597+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:52.059 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:52.058+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:52.549 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:52.548+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:53.075 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:53.074+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:53.546 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:53.545+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:54.114 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:54.113+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:54.576 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:54.575+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:55.073 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:55.072+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:55.613 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:55.612+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:56.084 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:56.084+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:56.625 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:56.624+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:57.076 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:57.075+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:57.646 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:57.645+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:58.098 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:58.096+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:58.630 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:58.629+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:59.102 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:31:59.100+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:31:59.637 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:31:59.635+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:00.080 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:00.079+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:00.618 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:00.616+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:01.057 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:01.055+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:01.645 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:01.643+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:02.035 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:02.033+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:02.653 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:02.651+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:03.009 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:03.007+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:03.643 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:03.641+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:03.960 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:03.959+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:04.634 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:04.632+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:05.011 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:05.009+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:05.639 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:05.637+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:05.975 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:05.974+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:06.660 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:06.658+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:06.940 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:06.938+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:07.615 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:07.613+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:07.964 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:07.962+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:08.621 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:08.620+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:08.941 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:08.939+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:09.585 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:09.583+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:09.922 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:09.920+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:10.572 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:10.570+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:10.921 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:10.919+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:11.584 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:11.582+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:11.923 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:11.921+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:12.599 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:12.598+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:12.915 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:12.913+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:13.621 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:13.619+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:13.880 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:13.878+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:14.588 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:14.586+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:14.854 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:14.852+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:15.545 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:15.543+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:15.896 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:15.894+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:16.568 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:16.566+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:16.924 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:16.922+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:17.546 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:17.544+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:17.937 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:17.935+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:18.534 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:18.532+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:18.984 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:18.982+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:19.561 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:19.559+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:19.975 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:19.973+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:20.573 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:20.571+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:20.957 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:20.955+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:21.616 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:21.615+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:21.941 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:21.939+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:22.616 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:22.614+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:22.956 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:22.954+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:23.631 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:23.629+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:23.935 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:23.933+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:24.609 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:24.607+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:24.958 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:24.957+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:25.616 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:25.614+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:25.927 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:25.925+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:26.570 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:26.568+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:26.906 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:26.904+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:27.559 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:27.557+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:27.903 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:27.901+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:28.540 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:28.538+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:28.892 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:28.890+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:29.496 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:29.494+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:29.894 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:29.892+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:30.530 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:30.529+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:30.856 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:30.854+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:31.550 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:31.548+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:31.838 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:31.836+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:32.544 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:32.543+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:32.856 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:32.854+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:33.511 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:33.509+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:33.816 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:33.814+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:34.545 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:34.543+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:34.842 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:34.840+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:35.566 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:35.564+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:35.824 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:35.822+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:36.575 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:36.574+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:36.777 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:36.776+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:37.571 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:37.570+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:37.787 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:37.786+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:38.548 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:38.547+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:38.830 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:38.830+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:39.524 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:39.523+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:39.831 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:39.830+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:40.543 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:40.542+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:40.856 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:40.855+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:41.524 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:41.524+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:41.864 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:41.863+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:42.485 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:42.484+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:42.819 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:42.818+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:43.494 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:43.493+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:43.830 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:43.829+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:44.454 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:44.453+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:44.873 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:44.872+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:45.412 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:45.411+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:45.921 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:45.920+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:46.445 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:46.444+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:46.903 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:46.902+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:47.407 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:47.406+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:47.872 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:47.871+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:48.424 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:48.424+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:48.842 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:48.841+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:49.385 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:49.384+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:49.873 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:49.872+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:50.355 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:50.355+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:50.873 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:50.872+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:51.359 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:51.358+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:51.836 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:51.835+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:52.368 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:52.367+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:52.856 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:52.855+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:53.329 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:53.328+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:53.882 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:53.881+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:54.296 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:54.296+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:54.845 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:54.844+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:55.299 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:55.298+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:55.822 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:55.821+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:56.349 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:56.347+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:56.844 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:56.843+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:57.331 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:57.329+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:57.888 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:57.886+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:58.340 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:58.338+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:58.885 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:58.883+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:59.330 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:32:59.328+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:32:59.895 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:32:59.893+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:00.341 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:00.339+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:00.897 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:00.896+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:01.381 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:01.379+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:01.947 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:01.945+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:02.416 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:02.415+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:02.965 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:02.963+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:03.437 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:03.435+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:04.011 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:04.009+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:04.397 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:04.395+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:04.963 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:04.962+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:05.424 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:05.422+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:06.012 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:06.010+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:06.440 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:06.438+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:07.052 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:07.050+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:07.465 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:07.464+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:08.059 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:08.057+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:08.495 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:08.493+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:09.092 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:09.090+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:09.541 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:09.539+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:10.133 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:10.131+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:10.493 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:10.491+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:11.087 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:11.085+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:11.444 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:11.442+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:12.106 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:12.105+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:12.486 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:12.485+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:13.071 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:13.069+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:13.520 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:13.519+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:14.091 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:14.089+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:14.548 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:14.546+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:15.098 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:15.096+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:15.568 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:15.567+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:16.109 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:16.107+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:16.602 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:16.600+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:17.117 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:17.115+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:17.603 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:17.601+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:18.096 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:18.094+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:18.579 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:18.577+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:19.118 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:19.117+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:19.606 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:19.604+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:20.121 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:20.119+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:20.589 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:20.587+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:21.160 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:21.159+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:21.582 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:21.580+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:22.196 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:22.194+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:22.575 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:22.573+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:23.184 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:23.182+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:23.572 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:23.570+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:24.209 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:24.207+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:24.570 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:24.569+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:25.203 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:25.201+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:25.547 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:25.545+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:26.240 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:26.238+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:26.512 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:26.511+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:27.205 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:27.203+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:27.486 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:27.485+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:28.231 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:28.229+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:28.517 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:28.513+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:29.272 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:29.270+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:29.470 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:29.468+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:30.241 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:30.240+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:30.486 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:30.485+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:31.245 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:31.243+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:31.476 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:31.474+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:32.264 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:32.262+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:32.427 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:32.426+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:33.237 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:33.236+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:33.451 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:33.450+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:34.207 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:34.206+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:34.462 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:34.460+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:35.174 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:35.173+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:35.465 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:35.464+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:36.197 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:36.196+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:36.436 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:36.435+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:37.213 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:37.212+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:37.471 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:37.471+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:38.214 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:38.214+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:38.465 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:38.464+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:39.223 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:39.222+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:39.512 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:39.511+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:40.232 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:40.231+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:40.479 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:40.478+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:41.241 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:41.240+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:41.432 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:41.432+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:42.195 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:42.195+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:42.482 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:42.482+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:43.172 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:43.171+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:43.518 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:43.518+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:44.191 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:44.191+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:44.500 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:44.499+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:45.162 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:45.160+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:45.539 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:45.538+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:46.199 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:46.198+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:46.527 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:46.526+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:47.195 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:47.194+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:47.539 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:47.538+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:48.215 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:48.215+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:48.561 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:48.560+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:49.206 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:49.206+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:49.555 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:49.554+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:50.169 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:50.168+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:50.568 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:50.567+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:51.161 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:51.160+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:51.603 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:51.602+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:52.210 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:52.210+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:52.589 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:52.588+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:53.168 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:53.168+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:53.576 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:53.576+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:54.189 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:54.188+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:54.615 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:54.614+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:55.223 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:55.221+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:55.614 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:55.613+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:56.194 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:56.192+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:56.641 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:56.639+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:57.186 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:57.185+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:57.685 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:57.683+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:58.208 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:58.207+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:58.656 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:58.655+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:59.186 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:33:59.184+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:33:59.632 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:33:59.630+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:00.163 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:00.161+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:00.640 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:00.638+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:01.199 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:01.197+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:01.666 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:01.664+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:02.190 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:02.189+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:02.691 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:02.689+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:03.212 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:03.210+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:03.709 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:03.707+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:04.222 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:04.220+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:04.672 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:04.671+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:05.265 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:05.263+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:05.695 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:05.693+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:06.263 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:06.261+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:06.736 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:06.734+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:07.261 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:07.259+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:07.725 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:07.723+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:08.308 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:08.306+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:08.769 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:08.767+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:09.290 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:09.288+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:09.780 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:09.778+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:10.306 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:10.304+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:10.807 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:10.805+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:11.267 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:11.265+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:11.842 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:11.840+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:12.242 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:12.240+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:12.822 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:12.821+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:13.261 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:13.259+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:13.792 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:13.790+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:14.262 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:14.260+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:14.771 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:14.769+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:15.311 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:15.309+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:15.783 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:15.781+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:16.287 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:16.285+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:16.758 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:16.757+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:17.299 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:17.297+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:17.786 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:17.784+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:18.312 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:18.310+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:18.835 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:18.834+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:19.307 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:19.305+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:19.813 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:19.811+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:20.325 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:20.323+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:20.767 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:20.765+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:21.375 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:21.373+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:21.816 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:21.814+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:22.385 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:22.383+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:22.810 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:22.808+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:23.433 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:23.432+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:23.837 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:23.835+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:24.425 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:24.423+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:24.849 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:24.847+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:25.462 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:25.460+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:25.807 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:25.805+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:26.438 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:26.436+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:26.802 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:26.801+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:27.390 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:27.388+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:27.822 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:27.820+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:28.387 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:28.385+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:28.808 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:28.806+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:29.347 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:29.345+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:29.827 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:29.825+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:30.354 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:30.352+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:30.827 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:30.826+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:31.327 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:31.325+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:31.851 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:31.849+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:32.374 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:32.372+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:32.812 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:32.811+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:33.361 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:33.360+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:33.846 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:33.845+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:34.338 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:34.337+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:34.875 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:34.874+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:35.313 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:35.313+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:35.891 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:35.890+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:36.326 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:36.325+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:36.855 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:36.854+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:37.282 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:37.281+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:37.855 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:37.854+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:38.255 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:38.254+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:38.839 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:38.838+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:39.235 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:39.235+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:39.818 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:39.817+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:40.271 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:40.271+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:40.787 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:40.787+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:41.254 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:41.254+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:41.740 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:41.739+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:42.247 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:42.246+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:42.735 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:42.734+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:43.245 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:43.244+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:43.693 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:43.692+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:44.224 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:44.223+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:44.678 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:44.677+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:45.194 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:45.193+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:45.700 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:45.699+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:46.210 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:46.209+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:46.688 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:46.688+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:47.215 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:47.214+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:47.648 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:47.647+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:48.239 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:48.238+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:48.678 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:48.677+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:49.258 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:49.257+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:49.642 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:49.641+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:50.277 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:50.276+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:50.635 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:50.634+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:51.277 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:51.276+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:51.682 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:51.681+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:52.273 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:52.273+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:52.702 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:52.701+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:53.287 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:53.286+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:53.744 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:53.742+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:54.263 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:54.261+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:54.706 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:54.704+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:55.266 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:55.264+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:55.754 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:55.752+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:56.262 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:56.260+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:56.717 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:56.715+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:57.223 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:57.221+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:57.728 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:57.726+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:58.221 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:58.219+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:58.684 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:58.682+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:59.262 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:34:59.260+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:34:59.725 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:34:59.723+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:00.274 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:00.272+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:00.677 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:00.675+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:01.232 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:01.230+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:01.725 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:01.723+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:02.214 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:02.212+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:02.693 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:02.692+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:03.254 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:03.252+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:03.689 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:03.687+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:04.216 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:04.214+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:04.718 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:04.717+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:05.267 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:05.265+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:05.740 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:05.738+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:06.300 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:06.298+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:06.704 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:06.702+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:07.263 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:07.261+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:07.671 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:07.669+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:08.245 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:08.243+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:08.679 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:08.678+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:09.224 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:09.222+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:09.691 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:09.690+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:10.230 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:10.229+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:10.664 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:10.662+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:11.213 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:11.211+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:11.674 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:11.673+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:12.240 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:12.238+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:12.656 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:12.654+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:13.213 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:13.211+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:13.631 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:13.630+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:14.240 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:14.238+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:14.625 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:14.623+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:15.247 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:15.245+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:15.664 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:15.662+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:16.295 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:16.293+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:16.677 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:16.675+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:17.247 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:17.245+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:17.712 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:17.711+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:18.261 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:18.259+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:18.722 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:18.720+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:19.254 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:19.251+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:19.695 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:19.693+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:20.208 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:20.206+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:20.671 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:20.670+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:21.189 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:21.187+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:21.664 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:21.662+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:22.162 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:22.160+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:22.658 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:22.656+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:23.114 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:23.112+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:23.704 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:23.702+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:24.108 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:24.106+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:24.669 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:24.668+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:25.105 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:25.103+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:25.715 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:25.713+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:26.064 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:26.063+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:26.724 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:26.722+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:27.107 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:27.105+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:27.734 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:27.732+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:28.139 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:28.138+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:28.745 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:28.743+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:29.147 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:29.146+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:29.711 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:29.709+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:30.147 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:30.145+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:30.724 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:30.722+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:31.180 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:31.179+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:31.760 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:31.759+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:32.166 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:32.165+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:32.736 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:32.735+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:33.204 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:33.203+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:33.762 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:33.761+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:34.177 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:34.177+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:34.763 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:34.762+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:35.225 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:35.224+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:35.802 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:35.801+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:36.255 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:36.254+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:36.818 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:36.817+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:37.216 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:37.215+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:37.812 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:37.811+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:38.188 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:38.187+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:38.773 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:38.772+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:39.211 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:39.210+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:39.822 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:39.821+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:40.246 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:40.245+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:40.803 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:40.802+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:41.201 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:41.200+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:41.794 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:41.793+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:42.187 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:42.187+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:42.757 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:42.756+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:43.163 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:43.162+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:43.736 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:43.735+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:44.120 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:44.120+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:44.766 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:44.765+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:45.126 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:45.125+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:45.735 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:45.734+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:46.089 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:46.088+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:46.738 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:46.737+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:47.137 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:47.136+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:47.716 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:47.715+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:48.150 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:48.149+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:48.683 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:48.682+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:49.107 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:49.106+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:49.701 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:49.700+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:50.157 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:50.156+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:50.669 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:50.668+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:51.130 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:51.130+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:51.646 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:51.644+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:52.156 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:52.154+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:52.625 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:52.623+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:53.185 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:53.183+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:53.627 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:53.625+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:54.220 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:54.218+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:54.656 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:54.654+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:55.203 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:55.201+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:55.673 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:55.672+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:56.210 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:56.208+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:56.700 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:56.699+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:57.169 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:57.167+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:57.685 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:57.683+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:58.179 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:58.178+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:58.650 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:58.649+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:59.156 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:35:59.154+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:35:59.614 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:35:59.612+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:00.133 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:00.131+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:00.621 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:00.617+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:01.169 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:01.167+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:01.610 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:01.608+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:02.181 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:02.180+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:02.644 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:02.642+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:03.193 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:03.191+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:03.669 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:03.667+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:04.192 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:04.190+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:04.641 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:04.639+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:05.197 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:05.195+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:05.638 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:05.637+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:06.214 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:06.213+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:06.611 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:06.609+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:07.207 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:07.205+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:07.567 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:07.566+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:08.222 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:08.220+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:08.557 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:08.555+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:09.226 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:09.225+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:09.548 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:09.546+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:10.217 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:10.215+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:10.525 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:10.523+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:11.198 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:11.196+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:11.548 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:11.546+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:12.155 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:12.153+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:12.538 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:12.536+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:13.116 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:13.114+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:13.583 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:13.582+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:14.122 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:14.120+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:14.575 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:14.574+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:15.080 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:15.078+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:15.604 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:15.602+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:16.104 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:16.102+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:16.575 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:16.573+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:17.059 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:17.057+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:17.565 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:17.563+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:18.087 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:18.085+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:18.606 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:18.604+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:19.068 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:19.066+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:19.618 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:19.616+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:20.103 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:20.102+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:20.642 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:20.640+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:21.154 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:21.152+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:21.656 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:21.654+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:22.168 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:22.166+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:22.663 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:22.661+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:23.163 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:23.161+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:23.624 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:23.622+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:24.159 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:24.157+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:24.601 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:24.599+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:25.158 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:25.156+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:25.612 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:25.610+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:26.178 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:26.176+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:26.603 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:26.602+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:27.212 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:27.210+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:27.628 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:27.626+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:28.169 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:28.167+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:28.643 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:28.642+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:29.167 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:29.165+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:29.689 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:29.688+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:30.176 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:30.175+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:30.664 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:30.663+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:31.178 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:31.177+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:31.714 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:31.713+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:32.226 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:32.225+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:32.680 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:32.679+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:33.258 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:33.257+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:33.710 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:33.710+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:34.301 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:34.300+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:34.662 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:34.661+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:35.320 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:35.319+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:35.675 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:35.675+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:36.276 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:36.275+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:36.636 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:36.635+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:37.282 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:37.281+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:37.668 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:37.667+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:38.274 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:38.273+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:38.659 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:38.658+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:39.289 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:39.288+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:39.704 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:39.704+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:40.325 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:40.324+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:40.710 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:40.710+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:41.372 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:41.371+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:41.687 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:41.686+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:42.387 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:42.386+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:42.686 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:42.685+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:43.345 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:43.344+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:43.701 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:43.700+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:44.333 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:44.332+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:44.745 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:44.744+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:45.382 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:45.381+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:45.713 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:45.712+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:46.379 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:46.379+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:46.746 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:46.745+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:47.352 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:47.351+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:47.755 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:47.755+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:48.389 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:48.388+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:48.789 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:48.788+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:49.406 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:49.405+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:49.823 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:49.822+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:50.362 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:50.360+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:50.828 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:50.826+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:51.384 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:51.382+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:51.810 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:51.808+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:52.435 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:52.432+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:52.846 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:52.845+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:53.462 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:53.460+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:53.849 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:53.847+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:54.494 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:54.492+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:54.870 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:54.868+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:55.464 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:55.462+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:55.892 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:55.890+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:56.456 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:56.454+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:56.885 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:56.883+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:57.463 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:57.461+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:57.914 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:57.912+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:58.456 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:58.454+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:58.938 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:58.936+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:59.482 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:36:59.480+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:36:59.986 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:36:59.984+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:00.469 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:00.467+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:01.002 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:01.000+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:01.492 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:01.490+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:01.954 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:01.952+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:02.524 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:02.522+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:02.916 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:02.914+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:03.554 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:03.552+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:03.946 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:03.944+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:04.540 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:04.538+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:04.957 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:04.956+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:05.536 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:05.534+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:05.909 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:05.907+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:06.577 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:06.575+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:06.894 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:06.892+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:07.618 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:07.616+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:07.848 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:07.847+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:08.595 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:08.592+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:08.846 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:08.844+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:09.555 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:09.553+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:09.823 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:09.821+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:10.584 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:10.582+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:10.865 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:10.863+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:11.604 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:11.602+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:11.861 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:11.859+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:12.612 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:12.611+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:12.814 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:12.812+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:13.579 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:13.578+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:13.770 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:13.768+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:14.600 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:14.598+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:14.753 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:14.752+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:15.624 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:15.622+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:15.936 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:15.780+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:16.660 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:16.659+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:16.804 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:16.802+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:17.676 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:17.675+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:17.760 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:17.758+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:18.641 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:18.639+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:18.785 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:18.783+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:19.674 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:19.672+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:19.764 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:19.762+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:20.651 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:20.649+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:20.756 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:20.754+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:21.693 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:21.692+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:21.801 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:21.799+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:22.701 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:22.700+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:22.775 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:22.773+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:23.719 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:23.717+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:23.734 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:23.732+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:24.702 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:24.700+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:24.775 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:24.773+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:25.727 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:25.725+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:25.821 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:25.819+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:26.748 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:26.746+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:26.837 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:26.835+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:27.727 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:27.726+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:27.862 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:27.861+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:28.724 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:28.723+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:28.871 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:28.870+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:29.704 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:29.704+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:29.857 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:29.857+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:30.747 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:30.747+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:30.836 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:30.835+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:31.761 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:31.757+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:31.807 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:31.807+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:32.710 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:32.709+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:32.807 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:32.807+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:33.690 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:33.689+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:33.771 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:33.771+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:34.731 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:34.730+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:34.812 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:34.811+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:35.748 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:35.747+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:35.846 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:35.845+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:36.744 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:36.743+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:36.824 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:36.823+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:37.721 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:37.720+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:37.787 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:37.786+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:38.767 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:38.766+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:38.834 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:38.833+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:39.733 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:39.732+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:39.801 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:39.801+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:40.729 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:40.729+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:40.762 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:40.761+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:41.759 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:41.759+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:41.802 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:41.801+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:42.754 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:42.753+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:42.824 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:42.823+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:43.754 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:43.753+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:43.872 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:43.871+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:44.741 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:44.740+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:44.835 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:44.833+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:45.751 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:45.751+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:45.830 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:45.829+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:46.723 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:46.723+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:46.879 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:46.879+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:47.771 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:47.771+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:47.843 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:47.842+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:48.813 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:48.811+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:48.815 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:48.813+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:49.822 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:49.821+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:49.849 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:49.847+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:50.868 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:50.866+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:50.898 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:50.897+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:51.877 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:51.875+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:51.882 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:51.880+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:52.841 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:52.840+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:52.910 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:52.908+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:53.822 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:53.820+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:53.954 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:53.952+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:54.824 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:54.822+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:54.949 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:54.947+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:55.855 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:55.853+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:55.984 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:55.982+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:56.858 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:56.856+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:56.981 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:56.979+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:57.895 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:57.893+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:57.978 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:57.976+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:58.850 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:58.848+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:58.979 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:58.977+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:59.828 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:37:59.826+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:37:59.936 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:37:59.935+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:00.812 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:00.811+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:00.891 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:00.889+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:01.779 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:01.778+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:01.908 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:01.906+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:02.740 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:02.738+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:02.915 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:02.914+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:03.698 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:03.696+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:03.871 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:03.869+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:04.736 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:04.734+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:04.842 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:04.840+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:05.778 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:05.776+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:05.846 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:05.845+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:06.739 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:06.738+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:06.834 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:06.832+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:07.722 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:07.720+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:07.851 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:07.849+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:08.726 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:08.724+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:08.886 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:08.884+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:09.764 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:09.762+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:09.894 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:09.892+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:10.814 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:10.812+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:10.899 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:10.898+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:11.845 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:11.844+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:11.921 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:11.919+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:12.843 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:12.842+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:12.954 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:12.952+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:13.893 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:13.892+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:13.915 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:13.914+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:14.864 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:14.862+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:14.883 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:14.881+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:15.832 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:15.830+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:15.852 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:15.850+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:16.817 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:16.816+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:16.883 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:16.882+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:17.834 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:17.833+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:17.899 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:17.897+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:18.830 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:18.828+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:18.853 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:18.852+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:19.820 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:19.818+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:19.892 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:19.891+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:20.857 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:20.855+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:20.895 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:20.894+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:21.826 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:21.824+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:21.917 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:21.916+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:22.797 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:22.795+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:22.873 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:22.872+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:23.804 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:23.802+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:23.865 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:23.864+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:24.819 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:24.817+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:24.843 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:24.843+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:25.796 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:25.794+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:25.889 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:25.887+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:26.760 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:26.760+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:26.844 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:26.843+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:27.805 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:27.804+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:27.818 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:27.818+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:28.829 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:28.829+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:28.863 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:28.862+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:29.790 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:29.789+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:29.893 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:29.893+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:30.761 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:30.761+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:30.891 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:30.891+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:31.776 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:31.775+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:31.912 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:31.911+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:32.815 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:32.815+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:32.863 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:32.862+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:33.836 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:33.835+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:33.896 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:33.896+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:34.830 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:34.830+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:34.888 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:34.887+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:35.845 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:35.844+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:35.937 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:35.936+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:36.835 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:36.834+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:36.943 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:36.942+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:37.880 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:37.879+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:37.929 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:37.929+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:38.856 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:38.855+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:38.942 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:38.941+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:39.822 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:39.821+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:39.962 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:39.962+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:40.782 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:40.781+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:40.916 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:40.915+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:41.822 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:41.821+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:41.952 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:41.952+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:42.814 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:42.813+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:42.941 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:42.940+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:43.825 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:43.824+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:43.939 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:43.938+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:44.842 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:44.841+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:44.900 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:44.899+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:45.854 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:45.854+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:45.883 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:45.882+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:46.856 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:46.854+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:46.899 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:46.897+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:47.896 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:47.894+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:47.949 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:47.948+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:48.872 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:48.870+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:48.996 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:48.994+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:49.837 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:49.836+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:49.954 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:49.952+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:50.796 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:50.794+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:50.946 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:50.944+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:51.767 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:51.766+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:51.898 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:51.896+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:52.738 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:52.736+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:52.859 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:52.857+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:53.712 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:53.710+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:53.834 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:53.832+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:54.745 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:54.744+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:54.835 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:54.833+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:55.722 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:55.721+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:55.874 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:55.872+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:56.768 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:56.766+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:56.912 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:56.910+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:57.801 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:57.799+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:57.916 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:57.914+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:58.782 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:58.780+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:58.938 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:58.937+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:59.748 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:38:59.746+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:38:59.896 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:38:59.894+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:00.771 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:00.770+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:00.932 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:00.930+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:01.750 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:01.748+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:01.979 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:01.978+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:02.735 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:02.732+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:02.942 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:02.941+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:03.772 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:03.771+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:03.928 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:03.926+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:04.763 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:04.762+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:04.905 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:04.903+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:05.730 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:05.729+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:05.903 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:05.902+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:06.706 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:06.704+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:06.932 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:06.931+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:07.728 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:07.726+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:07.923 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:07.921+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:08.705 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:08.704+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:08.918 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:08.916+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:09.729 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:09.727+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:09.886 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:09.884+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:10.776 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:10.774+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:10.879 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:10.877+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:11.796 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:11.795+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:11.868 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:11.866+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:12.764 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:12.762+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:12.894 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:12.892+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:13.735 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:13.734+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:13.853 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:13.851+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:14.740 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:14.738+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:14.830 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:14.828+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:15.691 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:15.689+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:15.813 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:15.811+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:16.671 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:16.669+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:16.862 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:16.860+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:17.701 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:17.699+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:17.873 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:17.871+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:18.745 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:18.743+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:18.900 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:18.898+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:19.752 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:19.750+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:19.923 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:19.921+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:20.744 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:20.742+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:20.916 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:20.914+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:21.757 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:21.755+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:21.945 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:21.943+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:22.739 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:22.737+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:22.929 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:22.927+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:23.747 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:23.746+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:23.894 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:23.893+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:24.741 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:24.740+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:24.852 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:24.852+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:25.774 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:25.773+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:25.877 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:25.876+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:26.806 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:26.806+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:26.877 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:26.876+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:27.795 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:27.795+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:27.889 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:27.888+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:28.843 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:28.842+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:28.890 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:28.890+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:29.812 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:29.811+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:29.883 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:29.882+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:30.784 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:30.783+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:30.872 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:30.871+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:31.736 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:31.735+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:31.872 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:31.871+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:32.706 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:32.706+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:32.845 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:32.845+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:33.712 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:33.711+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:33.844 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:33.843+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:34.707 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:34.706+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:34.817 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:34.816+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:35.757 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:35.756+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:35.786 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:35.785+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:36.716 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:36.716+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:36.809 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:36.808+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:37.683 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:37.682+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:37.782 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:37.781+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:38.693 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:38.692+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:38.832 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:38.831+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:39.727 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:39.726+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:39.830 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:39.830+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:40.760 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:40.760+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:40.802 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:40.801+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:41.733 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:41.732+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:41.823 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:41.822+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:42.742 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:42.741+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:42.803 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:42.802+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:43.780 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:43.779+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:43.788 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:43.788+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:44.758 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:44.756+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:44.816 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:44.815+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:45.712 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:45.710+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:45.776 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:45.774+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:46.759 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:46.757+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:46.781 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:46.779+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:47.731 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:47.729+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:47.768 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:47.766+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:48.717 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:48.715+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:48.759 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:48.757+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:49.688 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:49.686+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:49.807 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:49.806+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:50.700 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:50.698+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:50.850 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:50.848+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:51.735 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:51.734+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:51.822 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:51.821+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:52.724 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:52.723+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:52.855 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:52.853+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:53.734 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:53.732+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:53.819 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:53.817+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:54.777 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:54.775+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:54.799 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:54.797+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:55.736 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:55.734+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:55.792 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:55.790+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:56.770 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:56.769+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:56.782 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:56.780+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:57.789 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:57.787+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:57.794 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:57.792+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:58.808 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:58.806+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:58.823 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:58.822+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:59.762 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:39:59.761+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:39:59.853 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:39:59.851+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:00.721 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:00.719+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:00.818 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:00.816+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:01.695 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:01.693+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:01.779 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:01.777+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:02.662 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:02.660+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:02.815 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:02.813+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:03.635 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:03.633+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:03.849 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:03.846+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:04.596 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:04.595+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:04.868 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:04.867+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:05.583 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:05.581+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:05.848 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:05.846+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:06.538 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:06.536+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:06.891 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:06.889+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:07.568 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:07.566+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:07.912 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:07.910+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:08.549 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:08.547+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:08.951 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:08.949+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:09.600 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:09.598+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:09.931 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:09.929+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:10.566 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:10.564+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:10.931 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:10.929+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:11.539 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:11.537+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:11.978 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:11.976+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:12.516 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:12.514+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:12.994 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:12.993+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:13.482 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:13.481+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:14.020 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:14.019+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:14.458 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:14.456+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:14.989 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:14.987+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:15.444 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:15.443+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:15.953 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:15.952+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:16.464 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:16.462+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:16.970 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:16.969+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:17.491 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:17.489+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:17.969 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:17.967+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:18.487 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:18.485+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:18.960 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:18.958+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:19.472 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:19.470+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:19.939 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:19.937+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:20.442 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:20.440+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:20.958 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:20.956+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:21.412 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:21.410+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:21.967 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:21.966+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:22.410 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:22.409+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:22.987 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:22.986+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:23.364 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:23.363+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:23.987 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:23.986+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:24.360 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:24.359+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:25.028 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:25.027+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:25.347 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:25.346+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:26.069 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:26.068+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:26.309 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:26.308+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:27.119 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:27.119+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:27.280 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:27.279+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:28.129 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:28.126+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:28.321 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:28.321+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:29.101 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:29.099+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:29.300 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:29.299+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:30.119 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:30.119+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:30.342 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:30.341+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:31.155 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:31.154+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:31.331 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:31.330+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:32.128 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:32.128+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:32.356 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:32.355+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:33.081 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:33.080+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:33.337 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:33.337+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:34.128 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:34.127+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:34.324 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:34.324+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:35.082 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:35.081+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:35.309 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:35.308+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:36.129 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:36.129+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:36.274 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:36.274+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:37.154 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:37.153+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:37.315 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:37.315+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:38.124 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:38.123+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:38.286 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:38.285+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:39.158 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:39.157+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:39.310 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:39.309+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:40.111 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:40.110+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:40.312 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:40.311+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:41.161 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:41.160+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:41.285 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:41.283+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:42.163 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:42.162+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:42.309 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:42.309+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:43.157 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:43.155+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:43.337 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:43.335+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:44.125 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:44.123+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:44.297 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:44.296+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:45.130 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:45.129+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:45.279 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:45.278+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:46.168 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:46.166+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:46.323 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:46.322+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:47.181 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:47.179+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:47.353 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:47.351+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:48.151 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:48.149+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:48.316 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:48.314+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:49.118 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:49.116+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:49.348 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:49.347+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:50.122 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:50.120+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:50.375 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:50.374+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:51.118 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:51.116+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:51.364 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:51.362+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:52.073 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:52.072+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:52.379 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:52.378+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:53.091 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:53.089+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:53.388 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:53.386+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:54.092 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:54.090+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:54.427 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:54.425+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:55.127 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:55.126+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:55.421 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:55.418+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:56.082 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:56.080+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:56.398 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:56.396+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:57.039 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:57.037+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:57.388 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:57.386+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:58.071 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:58.069+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:58.426 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:58.425+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:59.111 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:40:59.109+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:40:59.412 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:40:59.410+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:00.125 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:00.124+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:00.391 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:00.390+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:01.153 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:01.151+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:01.354 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:01.353+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:02.180 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:02.178+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:02.390 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:02.388+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:03.148 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:03.146+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:03.432 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:03.431+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:04.103 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:04.101+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:04.404 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:04.402+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:05.101 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:05.099+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:05.420 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:05.418+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:06.054 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:06.052+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:06.392 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:06.390+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:07.024 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:07.022+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:07.347 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:07.345+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:08.063 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:08.061+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:08.397 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:08.395+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:09.102 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:09.100+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:09.353 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:09.351+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:10.080 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:10.078+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:10.360 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:10.358+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:11.032 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:11.030+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:11.323 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:11.322+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:12.032 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:12.030+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:12.296 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:12.294+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:12.999 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:12.997+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:13.273 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:13.271+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:13.964 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:13.962+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:14.316 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:14.314+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:15.007 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:15.005+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:15.286 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:15.284+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:15.986 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:15.985+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:16.289 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:16.287+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:17.028 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:17.026+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:17.295 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:17.293+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:18.039 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:18.037+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:18.272 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:18.271+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:19.036 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:19.034+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:19.233 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:19.231+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:20.056 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:20.054+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:20.186 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:20.184+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:21.025 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:21.025+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:21.216 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:21.215+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:22.047 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:22.046+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:22.262 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:22.262+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:23.060 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:23.059+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:23.229 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:23.228+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:24.015 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:24.014+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:24.182 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:24.181+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:25.052 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:25.051+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:25.223 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:25.222+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:26.055 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:26.054+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:26.231 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:26.230+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:27.073 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:27.072+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:27.267 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:27.266+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:28.083 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:28.082+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:28.225 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:28.224+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:29.107 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:29.107+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:29.209 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:29.209+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:30.063 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:30.062+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:30.217 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:30.217+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:31.045 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:31.044+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:31.212 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:31.212+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:32.039 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:32.038+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:32.235 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:32.235+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:33.067 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:33.067+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:33.260 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:33.259+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:34.084 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:34.083+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:34.257 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:34.256+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:35.093 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:35.092+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:35.254 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:35.253+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:36.107 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:36.106+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:36.288 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:36.287+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:37.102 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:37.102+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:37.309 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:37.309+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:38.089 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:38.088+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:38.319 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:38.319+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:39.122 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:39.121+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:39.352 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:39.350+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:40.106 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:40.105+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:40.346 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:40.345+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:41.069 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:41.068+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:41.313 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:41.311+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:42.102 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:42.100+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:42.276 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:42.274+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:43.072 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:43.070+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:43.256 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:43.254+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:44.084 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:44.082+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:44.236 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:44.234+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:45.094 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:45.092+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:45.205 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:45.204+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:46.098 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:46.096+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:46.181 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:46.180+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:47.113 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:47.110+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:47.196 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:47.194+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:48.129 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:48.127+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:48.235 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:48.234+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:49.086 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:49.085+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:49.263 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:49.261+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:50.065 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:50.063+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:50.270 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:50.268+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:51.026 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:51.025+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:51.259 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:51.258+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:52.047 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:52.046+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:52.226 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:52.225+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:53.016 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:53.014+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:53.226 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:53.224+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:53.991 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:53.989+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:54.207 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:54.205+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:55.017 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:55.015+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:55.158 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:55.156+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:55.982 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:55.980+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:56.163 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:56.162+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:56.942 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:56.940+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:57.161 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:57.159+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:57.967 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:57.966+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:58.118 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:58.116+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:58.948 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:58.947+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:59.130 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:41:59.128+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:41:59.953 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:41:59.951+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:00.171 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:00.170+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:00.987 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:00.985+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:01.148 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:01.146+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:01.950 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:01.948+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:02.139 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:02.138+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:02.902 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:02.900+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:03.117 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:03.115+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:03.898 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:03.896+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:04.095 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:04.093+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:04.917 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:04.915+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:05.085 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:05.083+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:05.916 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:05.915+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:06.105 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:06.103+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:06.911 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:06.910+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:07.119 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:07.118+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:07.946 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:07.945+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:08.163 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:08.161+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:08.951 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:08.950+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:09.195 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:09.194+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:09.938 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:09.936+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:10.151 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:10.150+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:10.979 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:10.977+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:11.161 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:11.159+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:12.006 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:12.004+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:12.186 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:12.184+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:12.992 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:12.991+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:13.157 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:13.155+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:14.025 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:14.023+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:14.137 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:14.135+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:14.999 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:14.998+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:15.162 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:15.160+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:16.045 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:16.043+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:16.192 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:16.191+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:16.998 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:16.996+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:17.163 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:17.161+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:17.960 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:17.958+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:18.209 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:18.209+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:18.957 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:18.956+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:19.178 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:19.178+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:19.939 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:19.938+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:20.144 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:20.144+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:20.916 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:20.915+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:21.194 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:21.193+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:21.903 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:21.902+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:22.231 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:22.230+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:22.934 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:22.933+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:23.226 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:23.225+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:23.955 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:23.954+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:24.258 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:24.257+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:24.919 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:24.919+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:25.242 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:25.241+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:25.966 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:25.965+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:26.203 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:26.203+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:27.004 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:27.004+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:27.172 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:27.171+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:27.967 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:27.967+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:28.200 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:28.200+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:28.989 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:28.988+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:29.182 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:29.181+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:29.985 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:29.984+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:30.211 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:30.210+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:30.999 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:30.998+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:31.233 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:31.233+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:32.010 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:32.009+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:32.281 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:32.280+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:32.985 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:32.984+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:33.285 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:33.284+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:34.033 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:34.032+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:34.307 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:34.307+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:35.076 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:35.075+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:35.259 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:35.259+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:36.037 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:36.037+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:36.276 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:36.275+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:37.032 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:37.031+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:37.311 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:37.310+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:38.039 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:38.038+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:38.304 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:38.302+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:39.081 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:39.079+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:39.278 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:39.276+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:40.110 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:40.108+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:40.292 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:40.290+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:41.160 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:41.158+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:41.341 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:41.340+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:42.139 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:42.138+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:42.323 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:42.321+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:43.142 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:43.140+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:43.304 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:43.302+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:44.133 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:44.132+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:44.326 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:44.324+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:45.116 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:45.114+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:45.322 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:45.320+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:46.132 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:46.130+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:46.348 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:46.346+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:47.110 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:47.108+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:47.385 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:47.383+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:48.112 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:48.111+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:48.358 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:48.356+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:49.135 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:49.134+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:49.377 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:49.375+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:50.163 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:50.161+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:50.387 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:50.386+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:51.114 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:51.112+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:51.425 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:51.422+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:52.104 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:52.102+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:52.392 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:52.390+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:53.104 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:53.103+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:53.394 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:53.392+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:54.132 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:54.130+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:54.354 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:54.352+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:55.136 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:55.134+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:55.348 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:55.346+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:56.117 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:56.116+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:56.341 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:56.339+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:57.102 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:57.100+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:57.379 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:57.377+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:58.117 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:58.116+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:58.427 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:58.425+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:59.141 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:42:59.139+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:42:59.443 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:42:59.442+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:00.169 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:00.168+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:00.436 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:00.435+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:01.192 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:01.191+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:01.486 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:01.485+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:02.184 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:02.182+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:02.524 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:02.522+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:03.223 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:03.222+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:03.547 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:03.545+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:04.262 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:04.260+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:04.525 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:04.524+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:05.254 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:05.252+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:05.548 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:05.546+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:06.274 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:06.273+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:06.589 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:06.588+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:07.241 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:07.239+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:07.579 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:07.577+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:08.195 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:08.193+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:08.569 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:08.568+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:09.185 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:09.183+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:09.592 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:09.590+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:10.160 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:10.158+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:10.543 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:10.542+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:11.120 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:11.119+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:11.528 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:11.526+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:12.111 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:12.110+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:12.546 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:12.545+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:13.070 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:13.069+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:13.555 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:13.553+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:14.100 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:14.098+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:14.535 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:14.534+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:15.078 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:15.077+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:15.559 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:15.558+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:16.087 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:16.086+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:16.602 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:16.601+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:17.081 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:17.080+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:17.640 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:17.639+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:18.068 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:18.067+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:18.685 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:18.684+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:19.068 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:19.067+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:19.717 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:19.716+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:20.075 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:20.075+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:20.684 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:20.683+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:21.082 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:21.081+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:21.653 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:21.652+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:22.048 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:22.047+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:22.672 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:22.672+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:23.073 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:23.072+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:23.648 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:23.647+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:24.034 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:24.033+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:24.679 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:24.679+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:25.001 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:25.000+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:25.684 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:25.683+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:25.964 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:25.963+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:26.675 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:26.674+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:26.998 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:26.998+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:27.641 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:27.641+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:27.955 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:27.955+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:28.633 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:28.633+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:28.930 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:28.929+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:29.653 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:29.652+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:29.910 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:29.909+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:30.653 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:30.652+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:30.912 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:30.911+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:31.698 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:31.698+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:31.937 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:31.937+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:32.732 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:32.731+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:32.925 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:32.924+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:33.694 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:33.693+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:33.922 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:33.921+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:34.694 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:34.693+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:34.967 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:34.966+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:35.706 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:35.704+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:35.931 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:35.930+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:36.673 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:36.671+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:36.975 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:36.974+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:37.679 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:37.677+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:37.958 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:37.956+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:38.686 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:38.684+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:38.918 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:38.916+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:39.642 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:39.640+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:39.967 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:39.965+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:40.607 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:40.605+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:41.003 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:41.000+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:41.636 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:41.635+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:41.981 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:41.979+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:42.661 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:42.659+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:42.967 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:42.965+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:43.618 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:43.616+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:44.015 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:44.013+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:44.588 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:44.586+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:45.001 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:45.000+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:45.543 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:45.542+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:45.989 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:45.987+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:46.585 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:46.584+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:46.952 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:46.950+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:47.590 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:47.588+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:47.945 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:47.944+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:48.626 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:48.624+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:48.971 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:48.969+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:49.595 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:49.594+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:49.978 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:49.976+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:50.638 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:50.636+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:50.959 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:50.957+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:51.618 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:51.616+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:51.977 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:51.975+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:52.665 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:52.663+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:53.017 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:53.015+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:53.657 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:53.655+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:54.023 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:54.021+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:54.661 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:54.660+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:55.034 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:55.032+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:55.696 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:55.694+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:56.001 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:55.999+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:56.671 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:56.669+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:57.038 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:57.037+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:57.717 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:57.715+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:58.017 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:58.015+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:58.733 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:58.731+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:59.063 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:43:59.061+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:43:59.764 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:43:59.763+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:00.026 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:00.025+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:00.736 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:00.735+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:01.067 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:01.066+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:01.694 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:01.692+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:02.066 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:02.064+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:02.702 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:02.700+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:03.043 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:03.041+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:03.667 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:03.666+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:04.073 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:04.072+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:04.641 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:04.639+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:05.025 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:05.023+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:05.670 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:05.669+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:06.060 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:06.058+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:06.627 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:06.625+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:07.068 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:07.066+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:07.671 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:07.670+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:08.074 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:08.073+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:08.654 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:08.653+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:09.074 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:09.069+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:09.648 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:09.645+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:10.119 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:10.117+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:10.662 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:10.660+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:11.131 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:11.130+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:11.691 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:11.689+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:12.174 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:12.172+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:12.685 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:12.683+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:13.160 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:13.159+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:13.641 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:13.640+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:14.188 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:14.187+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:14.686 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:14.686+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:15.210 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:15.210+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:15.695 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:15.694+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:16.204 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:16.203+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:16.742 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:16.741+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:17.224 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:17.224+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:17.727 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:17.726+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:18.263 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:18.263+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:18.722 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:18.720+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:19.273 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:19.272+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:19.771 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:19.770+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:20.249 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:20.248+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:20.764 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:20.764+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:21.239 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:21.238+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:21.768 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:21.767+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:22.273 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:22.272+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:22.804 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:22.803+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:23.268 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:23.267+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:23.815 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:23.815+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:24.240 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:24.239+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:24.853 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:24.852+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:25.279 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:25.278+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:25.858 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:25.858+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:26.239 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:26.238+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:26.885 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:26.885+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:27.246 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:27.246+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:27.879 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:27.878+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:28.236 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:28.235+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:28.892 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:28.891+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:29.193 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:29.192+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:29.932 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:29.931+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:30.162 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:30.161+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:30.926 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:30.925+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:31.206 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:31.206+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:31.890 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:31.889+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:32.167 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:32.166+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:32.941 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:32.940+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:33.127 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:33.124+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:33.908 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:33.907+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:34.100 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:34.098+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:34.861 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:34.859+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:35.056 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:35.055+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:35.864 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:35.863+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:36.036 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:36.034+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:36.874 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:36.872+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:37.004 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:37.002+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:37.866 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:37.864+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:38.046 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:38.044+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:38.877 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:38.876+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:39.070 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:39.068+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:39.904 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:39.902+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:40.078 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:40.076+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:40.874 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:40.872+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:41.100 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:41.098+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:41.892 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:41.890+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:42.075 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:42.073+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:42.871 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:42.869+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:43.028 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:43.027+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:43.907 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:43.906+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:44.022 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:44.020+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:44.941 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:44.939+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:44.975 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:44.973+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:45.919 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:45.917+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:45.937 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:45.935+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:46.920 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:46.918+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:46.948 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:46.947+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:47.967 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:47.965+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:47.983 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:47.982+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:48.960 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:48.958+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:48.983 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:48.981+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:49.962 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:49.960+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:49.986 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:49.984+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:50.948 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:50.946+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:50.981 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:50.979+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:51.913 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:51.911+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:51.973 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:51.972+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:52.912 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:52.911+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:52.991 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:52.990+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:53.946 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:53.945+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:53.966 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:53.964+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:54.937 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:54.936+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:54.973 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:54.971+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:55.926 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:55.924+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:55.976 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:55.974+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:56.948 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:56.946+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:56.984 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:56.982+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:57.916 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:57.914+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:57.971 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:57.970+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:58.880 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:58.879+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:58.948 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:58.947+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:59.881 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:44:59.879+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:44:59.959 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:44:59.957+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:00.872 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:00.871+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:00.968 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:00.966+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:01.875 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:01.873+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:01.965 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:01.963+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:02.904 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:02.902+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:02.918 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:02.917+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:03.913 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:03.911+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:03.915 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:03.913+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:04.893 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:04.891+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:04.949 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:04.947+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:05.899 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:05.898+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:05.994 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:05.993+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:06.895 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:06.893+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:06.964 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:06.963+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:07.868 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:07.867+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:07.965 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:07.963+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:08.840 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:08.839+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:08.944 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:08.942+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:09.818 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:09.816+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:09.906 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:09.905+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:10.789 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:10.789+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:10.894 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:10.893+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:11.816 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:11.815+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:11.931 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:11.930+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:12.851 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:12.850+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:12.972 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:12.971+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:13.897 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:13.896+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:13.927 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:13.926+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:14.902 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:14.901+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:14.922 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:14.921+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:15.879 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:15.878+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:15.903 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:15.902+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:16.882 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:16.881+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:16.898 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:16.897+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:17.902 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:17.901+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:17.923 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:17.923+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:18.896 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:18.895+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:18.912 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:18.911+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:19.860 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:19.859+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:19.937 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:19.937+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:20.817 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:20.817+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:20.895 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:20.895+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:21.799 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:21.798+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:21.928 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:21.927+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:22.779 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:22.779+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:22.903 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:22.902+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:23.786 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:23.785+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:23.902 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:23.901+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:24.791 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:24.790+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:24.909 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:24.908+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:25.830 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:25.829+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:25.959 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:25.958+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:26.849 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:26.848+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:26.917 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:26.917+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:27.835 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:27.834+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:27.891 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:27.890+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:28.864 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:28.863+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:28.919 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:28.918+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:29.863 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:29.862+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:29.939 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:29.938+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:30.832 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:30.830+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:30.904 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:30.902+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:31.808 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:31.806+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:31.875 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:31.874+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:32.793 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:32.789+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:32.847 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:32.846+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:33.797 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:33.796+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:33.815 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:33.814+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:34.806 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:34.805+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:34.812 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:34.810+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:35.808 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:35.806+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:35.814 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:35.812+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:36.782 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:36.780+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:36.862 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:36.860+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:37.790 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:37.789+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:37.879 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:37.878+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:38.748 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:38.746+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:38.904 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:38.902+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:39.784 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:39.783+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:39.873 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:39.872+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:40.796 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:40.794+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:40.836 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:40.834+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:41.814 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:41.812+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:41.867 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:41.865+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:42.780 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:42.778+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:42.902 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:42.900+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:43.735 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:43.734+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:43.941 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:43.940+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:44.706 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:44.704+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:44.902 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:44.900+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:45.671 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:45.670+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:45.857 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:45.856+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:46.681 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:46.679+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:46.865 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:46.864+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:47.707 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:47.705+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:47.842 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:47.841+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:48.744 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:48.742+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:48.843 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:48.841+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:49.702 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:49.700+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:49.873 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:49.871+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:50.653 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:50.651+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:50.851 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:50.849+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:51.652 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:51.651+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:51.832 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:51.830+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:52.622 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:52.621+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:52.803 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:52.802+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:53.630 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:53.628+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:53.852 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:53.850+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:54.616 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:54.615+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:54.831 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:54.829+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:55.570 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:55.569+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:55.805 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:55.804+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:56.547 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:56.545+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:56.854 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:56.852+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:57.550 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:57.548+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:57.874 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:57.872+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:58.542 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:58.540+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:58.836 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:58.834+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:59.525 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:45:59.523+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:45:59.812 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:45:59.811+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:00.505 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:00.502+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:00.826 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:00.825+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:01.503 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:01.501+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:01.835 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:01.834+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:02.468 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:02.466+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:02.804 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:02.802+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:03.432 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:03.430+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:03.801 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:03.799+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:04.446 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:04.445+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:04.808 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:04.806+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:05.448 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:05.447+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:05.833 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:05.832+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:06.435 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:06.430+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:06.845 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:06.843+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:07.411 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:07.409+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:07.843 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:07.841+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:08.421 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:08.420+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:08.835 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:08.834+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:09.435 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:09.434+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:09.812 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:09.811+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:10.457 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:10.456+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:10.808 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:10.807+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:11.417 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:11.416+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:11.763 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:11.763+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:12.395 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:12.394+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:12.773 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:12.772+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:13.433 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:13.432+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:13.799 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:13.798+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:14.445 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:14.444+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:14.818 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:14.817+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:15.402 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:15.401+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:15.851 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:15.851+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:16.396 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:16.395+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:16.803 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:16.802+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:17.430 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:17.430+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:17.813 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:17.812+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:18.417 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:18.417+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:18.783 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:18.782+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:19.451 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:19.450+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:19.762 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:19.761+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:20.413 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:20.412+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:20.726 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:20.725+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:21.365 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:21.364+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:21.733 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:21.732+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:22.358 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:22.357+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:22.718 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:22.717+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:23.362 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:23.361+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:23.757 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:23.756+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:24.365 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:24.364+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:24.785 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:24.784+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:25.384 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:25.383+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:25.798 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:25.797+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:26.337 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:26.336+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:26.749 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:26.748+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:27.386 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:27.386+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:27.752 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:27.751+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:28.369 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:28.368+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:28.749 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:28.747+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:29.406 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:29.405+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:29.701 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:29.699+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:30.418 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:30.416+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:30.688 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:30.686+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:31.383 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:31.381+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:31.653 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:31.651+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:32.411 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:32.409+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:32.697 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:32.695+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:33.461 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:33.459+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:33.724 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:33.722+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:34.434 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:34.432+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:34.714 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:34.712+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:35.429 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:35.427+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:35.679 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:35.677+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:36.420 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:36.419+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:36.662 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:36.660+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:37.461 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:37.460+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:37.659 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:37.657+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:38.503 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:38.501+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:38.612 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:38.611+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:39.551 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:39.549+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:39.563 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:39.562+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:40.567 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:40.565+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:40.577 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:40.575+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:41.574 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:41.573+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:41.596 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:41.595+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:42.543 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:42.541+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:42.599 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:42.598+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:43.585 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:43.584+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:43.624 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:43.623+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:44.584 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:44.583+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:44.601 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:44.599+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:45.555 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:45.553+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:45.560 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:45.559+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:46.544 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:46.542+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:46.573 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:46.571+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:47.521 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:47.519+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:47.610 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:47.608+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:48.499 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:48.497+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:48.621 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:48.620+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:49.493 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:49.491+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:49.578 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:49.577+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:50.516 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:50.515+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:50.599 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:50.597+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:51.496 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:51.495+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:51.562 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:51.560+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:52.468 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:52.466+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:52.589 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:52.587+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:53.452 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:53.451+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:53.605 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:53.604+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:54.476 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:54.474+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:54.598 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:54.596+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:55.520 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:55.519+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:55.620 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:55.618+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:56.536 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:56.534+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:56.593 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:56.591+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:57.491 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:57.489+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:57.604 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:57.603+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:58.448 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:58.446+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:58.652 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:58.651+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:59.444 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:46:59.442+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:46:59.659 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:46:59.657+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:00.468 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:00.466+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:00.614 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:00.612+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:01.437 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:01.435+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:01.595 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:01.593+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:02.463 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:02.462+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:02.549 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:02.547+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:03.416 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:03.415+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:03.501 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:03.500+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:04.460 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:04.458+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:04.460 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:04.458+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:05.470 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:05.469+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:05.508 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:05.506+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:06.456 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:06.455+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:06.482 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:06.482+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:07.411 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:07.411+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:07.499 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:07.498+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:08.396 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:08.395+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:08.543 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:08.543+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:09.431 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:09.430+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:09.518 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:09.518+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:10.436 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:10.436+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:10.511 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:10.510+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:11.434 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:11.434+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:11.547 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:11.546+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:12.390 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:12.389+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:12.561 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:12.561+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:13.420 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:13.419+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:13.514 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:13.513+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:14.436 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:14.435+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:14.470 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:14.469+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:15.396 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:15.395+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:15.453 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:15.452+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:16.379 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:16.378+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:16.483 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:16.483+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:17.405 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:17.404+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:17.526 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:17.526+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:18.413 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:18.412+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:18.562 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:18.561+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:19.437 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:19.436+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:19.612 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:19.612+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:20.480 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:20.479+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:20.659 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:20.658+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:21.434 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:21.433+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:21.614 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:21.613+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:22.482 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:22.482+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:22.574 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:22.574+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:23.434 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:23.433+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:23.545 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:23.544+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:24.430 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:24.429+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:24.546 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:24.545+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:25.469 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:25.468+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:25.538 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:25.537+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:26.468 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:26.466+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:26.589 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:26.587+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:27.428 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:27.427+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:27.618 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:27.616+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:28.477 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:28.475+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:28.591 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:28.589+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:29.430 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:29.429+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:29.603 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:29.601+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:30.408 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:30.407+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:30.612 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:30.610+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:31.439 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:31.437+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:31.607 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:31.606+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:32.473 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:32.471+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:32.608 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:32.607+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:33.475 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:33.474+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:33.637 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:33.636+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:34.435 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:34.433+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:34.613 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:34.610+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:35.460 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:35.458+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:35.616 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:35.614+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:36.442 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:36.441+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:36.615 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:36.613+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:37.429 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:37.428+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:37.590 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:37.588+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:38.415 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:38.413+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:38.548 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:38.546+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:39.423 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:39.421+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:39.537 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:39.536+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:40.469 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:40.468+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:40.561 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:40.559+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:41.434 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:41.432+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:41.563 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:41.561+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:42.453 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:42.452+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:42.555 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:42.553+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:43.472 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:43.470+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:43.557 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:43.555+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:44.465 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:44.463+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:44.544 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:44.543+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:45.493 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:45.491+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:45.499 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:45.497+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:46.454 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:46.453+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:46.530 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:46.528+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:47.406 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:47.404+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:47.527 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:47.525+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:48.420 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:48.418+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:48.573 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:48.571+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:49.435 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:49.433+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:49.610 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:49.587+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:50.451 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:50.450+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:50.569 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:50.567+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:51.487 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:51.485+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:51.587 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:51.585+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:52.491 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:52.490+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:52.582 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:52.581+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:53.541 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:53.539+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:53.623 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:53.622+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:54.589 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:54.587+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:54.663 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:54.661+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:55.600 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:55.598+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:55.676 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:55.674+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:56.565 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:56.563+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:56.628 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:56.626+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:57.550 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:57.548+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:57.592 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:57.591+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:58.516 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:58.514+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:58.566 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:58.565+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:59.534 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:47:59.532+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:47:59.569 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:47:59.568+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:00.534 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:00.532+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:00.550 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:00.549+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:01.521 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:01.519+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:01.550 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:01.549+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:02.477 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:02.476+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:02.543 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:02.542+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:03.511 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:03.509+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:03.534 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:03.533+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:04.499 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:04.498+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:04.524 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:04.524+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:05.513 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:05.512+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:05.529 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:05.528+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:06.518 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:06.517+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:06.537 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:06.537+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:07.541 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:07.540+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:07.586 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:07.585+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:08.512 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:08.512+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:08.591 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:08.590+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:09.496 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:09.495+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:09.557 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:09.556+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:10.515 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:10.514+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:10.515 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:10.515+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:11.513 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:11.512+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:11.538 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:11.538+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:12.502 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:12.501+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:12.526 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:12.526+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:13.515 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:13.514+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:13.548 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:13.547+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:14.490 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:14.489+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:14.524 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:14.523+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:15.441 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:15.441+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:15.552 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:15.552+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:16.486 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:16.485+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:16.531 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:16.530+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:17.485 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:17.485+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:17.517 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:17.517+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:18.449 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:18.449+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:18.485 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:18.485+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:19.411 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:19.411+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:19.458 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:19.457+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:20.393 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:20.393+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:20.419 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:20.419+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:21.437 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:21.437+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:21.438 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:21.437+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:22.414 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:22.413+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:22.480 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:22.479+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:23.451 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:23.450+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:23.527 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:23.527+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:24.465 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:24.463+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:24.538 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:24.536+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:25.426 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:25.425+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:25.509 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:25.508+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:26.455 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:26.453+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:26.503 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:26.502+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:27.433 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:27.432+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:27.526 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:27.524+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:28.428 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:28.426+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:28.500 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:28.498+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:29.455 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:29.453+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:29.489 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:29.488+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:30.407 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:30.406+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:30.461 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:30.460+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:31.432 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:31.430+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:31.457 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:31.455+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:32.423 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:32.421+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:32.501 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:32.499+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:33.452 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:33.451+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:33.454 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:33.452+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:34.429 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:34.427+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:34.457 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:34.455+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:35.433 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:35.432+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:35.497 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:35.496+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:36.409 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:36.408+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:36.476 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:36.474+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:37.432 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:37.431+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:37.497 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:37.495+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:38.472 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:38.470+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:38.519 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:38.517+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:39.456 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:39.454+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:39.547 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:39.546+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:40.432 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:40.431+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:40.534 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:40.532+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:41.478 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:41.477+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:41.574 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:41.573+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:42.483 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:42.481+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:42.610 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:42.608+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:43.434 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:43.433+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:43.636 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:43.634+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:44.452 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:44.450+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:44.615 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:44.614+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:45.412 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:45.410+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:45.630 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:45.628+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:46.454 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:46.452+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:46.621 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:46.620+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:47.483 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:47.481+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:47.632 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:47.630+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:48.459 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:48.457+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:48.623 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:48.621+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:49.492 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:49.491+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:49.625 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:49.624+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:50.491 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:50.489+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:50.604 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:50.602+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:51.463 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:51.462+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:51.608 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:51.606+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:52.432 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:52.430+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:52.561 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:52.559+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:53.451 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:53.449+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:53.525 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:53.523+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:54.472 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:54.470+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:54.561 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:54.559+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:55.499 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:55.498+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:55.531 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:55.530+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:56.475 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:56.473+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:56.530 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:56.529+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:57.453 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:57.452+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:57.490 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:57.488+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:58.460 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:58.458+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:58.531 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:58.530+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:59.448 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:48:59.446+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:48:59.485 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:48:59.481+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:00.425 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:00.424+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:00.527 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:00.526+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:01.447 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:01.446+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:01.566 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:01.564+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:02.474 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:02.473+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:02.542 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:02.541+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:03.429 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:03.428+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:03.590 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:03.590+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:04.469 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:04.468+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:04.546 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:04.545+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:05.449 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:05.449+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:05.499 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:05.498+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:06.472 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:06.472+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:06.497 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:06.496+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:07.460 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:07.459+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:07.464 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:07.464+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:08.422 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:08.421+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:08.456 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:08.455+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:09.420 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:09.419+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:09.488 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:09.487+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:10.442 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:10.442+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:10.488 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:10.488+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:11.414 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:11.413+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:11.442 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:11.442+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:12.453 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:12.452+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:12.470 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:12.470+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:13.461 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:13.460+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:13.490 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:13.490+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:14.471 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:14.470+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:14.493 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:14.493+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:15.477 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:15.476+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:15.508 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:15.508+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:16.491 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:16.490+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:16.507 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:16.507+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:17.498 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:17.497+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:17.510 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:17.510+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:18.473 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:18.472+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:18.533 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:18.531+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:19.502 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:19.501+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:19.557 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:19.554+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:20.502 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:20.502+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:20.532 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:20.532+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:21.458 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:21.458+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:21.545 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:21.544+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:22.477 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:22.475+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:22.583 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:22.582+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:23.470 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:23.469+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:23.616 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:23.615+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:24.434 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:24.432+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:24.625 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:24.623+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:25.434 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:25.432+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:25.657 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:25.655+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:26.463 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:26.462+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:26.610 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:26.609+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:27.456 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:27.455+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:27.572 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:27.571+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:28.452 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:28.450+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:28.543 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:28.541+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:29.421 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:29.419+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:29.553 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:29.551+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:30.468 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:30.466+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:30.598 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:30.597+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:31.428 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:31.427+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:31.633 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:31.631+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:32.455 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:32.453+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:32.646 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:32.644+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:33.455 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:33.453+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:33.600 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:33.598+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:34.473 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:34.471+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:34.581 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:34.580+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:35.480 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:35.478+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:35.629 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:35.627+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:36.509 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:36.507+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:36.640 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:36.638+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:37.556 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:37.555+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:37.640 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:37.639+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:38.540 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:38.538+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:38.670 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:38.669+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:39.554 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:39.552+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:39.680 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:39.679+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:40.560 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:40.559+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:40.647 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:40.645+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:41.580 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:41.578+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:41.652 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:41.651+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:42.538 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:42.536+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:42.618 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:42.616+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:43.554 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:43.552+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:43.611 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:43.610+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:44.589 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:44.588+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:44.607 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:44.606+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:45.610 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:45.608+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:45.628 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:45.627+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:46.595 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:46.594+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:46.604 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:46.602+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:47.638 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:47.637+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:47.647 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:47.645+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:48.623 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:48.621+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:48.656 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:48.654+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:49.612 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:49.610+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:49.643 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:49.641+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:50.593 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:50.591+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:50.646 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:50.644+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:51.567 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:51.565+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:51.637 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:51.635+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:52.542 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:52.540+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:52.637 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:52.635+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:53.533 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:53.531+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:53.643 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:53.641+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:54.546 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:54.545+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:54.675 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:54.674+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:55.562 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:55.560+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:55.726 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:55.724+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:56.576 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:56.574+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:56.712 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:56.711+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:57.567 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:57.565+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:57.718 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:57.716+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:58.552 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:58.550+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:58.765 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:58.763+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:59.582 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:49:59.581+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:49:59.811 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:49:59.810+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:00.597 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:00.596+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:00.797 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:00.796+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:01.632 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:01.632+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:01.808 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:01.807+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:02.596 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:02.595+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:02.829 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:02.828+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:03.619 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:03.618+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:03.781 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:03.781+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:04.621 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:04.620+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:04.829 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:04.828+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:05.604 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:05.604+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:05.793 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:05.792+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:06.629 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:06.628+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:06.803 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:06.802+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:07.586 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:07.585+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:07.811 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:07.810+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:08.636 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:08.636+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:08.788 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:08.788+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:09.636 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:09.636+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:09.784 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:09.783+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:10.667 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:10.666+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:10.807 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:10.806+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:11.696 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:11.696+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:11.818 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:11.817+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:12.679 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:12.678+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:12.778 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:12.778+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:13.718 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:13.718+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:13.743 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:13.742+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:14.708 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:14.708+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:14.749 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:14.748+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:15.673 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:15.673+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:15.701 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:15.700+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:16.695 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:16.694+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:16.723 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:16.723+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:17.721 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:17.720+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:17.726 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:17.726+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:18.706 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:18.706+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:18.713 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:18.713+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:19.663 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:19.661+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:19.751 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:19.749+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:20.652 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:20.651+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:20.784 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:20.783+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:21.687 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:21.686+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:21.804 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:21.802+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:22.642 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:22.640+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:22.772 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:22.770+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:23.663 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:23.661+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:23.729 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:23.726+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:24.666 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:24.664+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:24.693 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:24.691+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:25.655 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:25.653+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:25.719 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:25.717+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:26.702 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:26.701+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:26.703 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:26.701+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:27.731 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:27.729+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:27.743 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:27.741+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:28.685 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:28.683+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:28.780 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:28.778+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:29.659 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:29.658+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:29.737 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:29.735+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:30.682 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:30.681+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:30.740 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:30.739+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:31.708 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:31.706+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:31.789 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:31.787+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:32.687 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:32.685+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:32.822 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:32.820+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:33.719 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:33.717+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:33.830 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:33.828+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:34.693 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:34.691+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:34.834 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:34.832+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:35.705 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:35.704+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:35.798 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:35.796+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:36.713 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:36.711+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:36.750 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:36.748+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:37.730 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:37.728+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:37.731 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:37.730+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:38.684 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:38.682+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:38.737 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:38.736+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:39.658 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:39.656+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:39.692 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:39.691+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:40.708 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:40.707+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:40.724 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:40.722+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:41.742 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:41.740+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:41.756 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:41.755+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:42.749 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:42.748+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:42.776 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:42.774+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:43.719 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:43.717+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:43.740 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:43.739+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:44.704 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:44.702+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:44.742 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:44.740+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:45.725 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:45.723+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:45.727 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:45.726+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:46.725 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:46.723+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:46.763 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:46.761+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:47.739 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:47.738+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:47.764 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:47.762+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:48.711 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:48.709+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:48.734 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:48.733+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:49.679 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:49.677+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:49.695 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:49.694+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:50.713 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:50.711+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:50.718 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:50.716+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:51.695 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:51.693+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:51.731 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:51.729+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:52.710 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:52.709+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:52.722 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:52.721+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:53.677 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:53.675+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:53.678 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:53.676+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:54.680 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:54.678+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:54.686 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:54.684+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:55.669 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:55.667+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:55.695 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:55.693+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:56.691 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:56.689+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:56.710 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:56.709+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:57.737 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:57.736+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:57.740 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:57.739+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:58.687 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:58.687+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:58.734 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:58.734+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:59.730 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:50:59.730+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:50:59.732 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:50:59.731+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:00.763 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:00.762+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:00.769 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:00.768+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:01.720 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:01.719+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:01.781 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:01.780+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:02.704 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:02.702+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:02.750 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:02.749+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:03.669 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:03.669+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:03.779 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:03.778+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:04.699 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:04.698+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:04.740 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:04.740+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:05.713 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:05.712+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:05.716 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:05.716+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:06.742 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:06.741+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:06.744 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:06.744+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:07.698 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:07.698+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:07.709 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:07.709+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:08.697 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:08.696+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:08.700 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:08.699+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:09.694 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:09.693+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:09.721 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:09.720+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:10.689 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:10.687+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:10.730 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:10.730+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:11.692 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:11.691+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:11.703 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:11.702+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:12.716 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:12.716+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:12.735 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:12.735+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:13.753 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:13.752+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:13.770 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:13.770+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:14.788 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:14.787+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:14.817 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:14.816+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:15.754 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:15.753+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:15.803 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:15.802+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:16.800 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:16.800+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:16.808 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:16.808+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:17.765 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:17.763+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:17.830 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:17.829+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:18.793 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:18.791+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:18.866 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:18.865+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:19.755 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:19.753+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:19.913 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:19.911+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:20.793 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:20.791+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:20.957 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:20.955+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:21.839 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:21.837+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:21.957 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:21.955+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:22.836 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:22.834+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:22.930 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:22.928+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:23.795 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:23.793+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:23.980 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:23.979+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:24.822 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:24.821+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:24.948 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:24.947+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:25.816 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:25.814+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:25.930 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:25.928+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:26.855 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:26.853+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:26.940 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:26.938+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:27.898 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:27.897+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:27.901 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:27.897+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:28.908 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:28.906+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:28.948 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:28.946+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:29.890 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:29.888+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:29.945 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:29.943+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:30.882 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:30.881+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:30.963 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:30.961+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:31.928 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:31.926+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:31.929 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:31.927+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:32.912 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:32.910+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:32.963 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:32.962+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:33.903 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:33.901+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:33.956 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:33.954+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:34.867 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:34.865+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:34.986 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:34.985+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:35.881 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:35.879+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:36.030 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:36.029+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:36.859 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:36.857+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:37.079 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:37.077+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:37.825 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:37.823+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:38.091 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:38.089+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:38.842 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:38.840+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:39.137 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:39.135+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:39.892 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:39.891+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:40.118 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:40.117+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:40.866 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:40.864+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:41.164 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:41.163+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:41.842 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:41.840+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:42.128 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:42.126+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:42.807 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:42.805+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:43.118 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:43.117+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:43.810 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:43.808+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:44.122 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:44.120+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:44.803 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:44.801+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:45.143 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:45.141+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:45.778 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:45.777+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:46.111 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:46.109+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:46.829 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:46.827+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:47.140 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:47.138+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:47.876 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:47.874+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:48.183 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:48.181+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:48.850 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:48.848+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:49.181 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:49.179+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:49.895 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:49.893+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:50.149 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:50.148+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:50.906 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:50.905+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:51.136 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:51.135+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:51.906 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:51.904+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:52.148 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:52.146+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:52.894 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:52.892+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:53.132 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:53.130+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:53.891 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:53.890+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:54.137 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:54.136+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:54.927 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:54.927+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:55.109 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:55.109+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:55.955 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:55.954+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:56.117 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:56.116+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:56.908 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:56.907+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:57.113 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:57.112+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:57.892 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:57.891+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:58.127 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:58.127+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:58.856 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:58.855+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:59.142 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:51:59.141+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:51:59.827 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:51:59.827+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:00.168 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:00.167+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:00.791 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:00.791+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:01.183 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:01.183+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:01.804 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:01.803+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:02.229 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:02.228+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:02.836 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:02.836+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:03.181 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:03.181+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:03.860 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:03.859+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:04.184 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:04.184+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:04.814 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:04.813+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:05.147 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:05.147+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:05.855 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:05.855+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:06.134 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:06.133+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:06.852 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:06.851+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:07.098 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:07.098+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:07.888 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:07.887+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:08.128 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:08.127+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:08.882 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:08.881+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:09.096 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:09.095+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:09.933 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:09.932+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:10.059 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:10.058+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:10.891 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:10.891+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:11.081 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:11.080+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:11.887 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:11.887+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:12.041 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:12.040+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:12.923 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:12.922+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:12.992 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:12.992+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:13.909 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:13.909+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:13.949 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:13.948+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:14.926 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:14.924+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:14.941 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:14.940+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:15.917 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:15.915+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:15.952 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:15.951+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:16.931 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:16.929+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:16.957 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:16.955+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:17.981 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:17.979+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:17.981 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:17.979+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:18.946 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:18.944+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:18.950 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:18.948+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:19.913 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:19.911+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:19.980 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:19.978+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:20.870 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:20.869+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:21.006 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:21.005+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:21.909 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:21.907+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:21.962 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:21.960+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:22.889 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:22.887+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:23.007 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:23.005+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:23.923 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:23.921+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:23.987 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:23.985+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:24.890 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:24.889+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:25.029 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:25.027+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:25.878 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:25.876+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:25.990 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:25.989+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:26.843 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:26.842+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:27.009 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:27.007+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:27.824 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:27.822+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:28.025 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:28.023+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:28.814 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:28.813+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:29.006 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:29.004+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:29.782 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:29.780+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:29.974 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:29.972+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:30.814 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:30.812+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:31.004 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:31.002+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:31.830 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:31.828+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:32.051 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:32.050+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:32.795 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:32.794+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:33.100 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:33.098+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:33.821 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:33.820+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:34.063 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:34.061+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:34.802 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:34.800+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:35.103 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:35.101+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:35.768 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:35.767+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:36.097 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:36.095+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:36.744 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:36.742+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:37.142 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:37.141+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:37.737 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:37.735+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:38.112 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:38.110+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:38.729 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:38.728+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:39.159 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:39.157+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:39.752 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:39.750+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:40.191 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:40.189+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:40.711 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:40.710+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:41.190 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:41.188+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:41.742 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:41.741+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:42.233 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:42.231+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:42.740 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:42.739+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:43.243 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:43.242+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:43.749 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:43.747+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:44.275 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:44.273+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:44.722 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:44.720+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:45.287 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:45.285+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:45.751 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:45.750+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:46.253 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:46.251+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:46.796 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:46.794+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:47.220 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:47.218+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:47.751 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:47.749+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:48.215 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:48.213+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:48.801 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:48.799+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:49.208 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:49.207+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:49.803 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:49.801+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:50.164 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:50.162+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:50.798 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:50.796+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:51.210 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:51.208+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:51.756 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:51.755+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:52.165 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:52.164+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:52.733 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:52.732+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:53.171 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:53.170+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:53.738 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:53.737+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:54.131 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:54.130+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:54.715 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:54.714+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:55.166 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:55.166+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:55.746 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:55.745+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:56.164 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:56.164+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:56.751 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:56.750+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:57.159 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:57.159+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:57.798 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:57.797+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:58.196 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:58.196+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:58.778 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:58.778+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:59.197 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:52:59.197+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:52:59.826 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:52:59.826+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:00.236 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:00.235+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:00.783 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:00.783+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:01.287 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:01.286+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:01.784 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:01.784+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:02.305 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:02.304+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:02.765 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:02.764+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:03.312 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:03.311+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:03.729 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:03.728+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:04.343 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:04.342+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:04.731 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:04.730+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:05.298 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:05.298+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:05.739 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:05.738+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:06.338 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:06.338+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:06.737 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:06.736+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:07.337 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:07.336+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:07.745 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:07.744+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:08.309 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:08.308+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:08.778 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:08.777+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:09.310 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:09.309+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:09.780 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:09.779+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:10.297 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:10.297+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:10.830 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:10.830+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:11.323 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:11.322+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:11.825 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:11.824+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:12.285 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:12.283+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:12.857 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:12.855+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:13.321 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:13.319+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:13.848 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:13.846+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:14.355 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:14.353+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:14.821 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:14.819+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:15.319 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:15.317+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:15.788 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:15.786+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:16.366 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:16.364+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:16.772 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:16.770+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:17.398 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:17.396+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:17.741 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:17.740+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:18.414 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:18.413+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:18.728 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:18.727+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:19.433 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:19.431+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:19.722 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:19.720+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:20.395 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:20.393+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:20.680 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:20.679+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:21.393 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:21.391+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:21.684 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:21.682+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:22.445 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:22.442+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:22.693 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:22.691+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:23.400 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:23.398+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:23.716 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:23.714+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:24.419 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:24.417+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:24.683 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:24.682+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:25.413 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:25.412+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:25.638 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:25.636+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:26.411 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:26.409+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:26.651 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:26.650+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:27.388 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:27.386+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:27.612 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:27.610+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:28.393 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:28.391+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:28.574 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:28.572+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:29.430 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:29.428+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:29.570 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:29.568+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:30.426 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:30.425+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:30.547 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:30.545+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:31.400 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:31.399+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:31.579 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:31.577+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:32.439 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:32.437+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:32.592 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:32.591+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:33.416 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:33.415+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:33.614 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:33.613+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:34.404 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:34.402+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:34.659 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:34.657+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:35.383 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:35.382+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:35.706 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:35.704+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:36.420 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:36.418+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:36.755 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:36.753+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:37.434 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:37.432+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:37.759 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:37.757+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:38.417 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:38.415+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:38.786 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:38.784+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:39.370 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:39.369+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:39.811 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:39.809+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:40.328 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:40.326+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:40.798 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:40.797+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:41.298 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:41.297+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:41.832 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:41.831+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:42.290 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:42.289+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:42.808 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:42.806+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:43.295 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:43.294+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:43.792 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:43.790+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:44.270 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:44.268+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:44.779 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:44.777+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:45.304 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:45.302+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:45.814 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:45.812+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:46.298 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:46.297+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:46.798 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:46.797+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:47.286 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:47.285+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:47.834 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:47.832+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:48.249 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:48.247+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:48.790 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:48.788+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:49.202 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:49.201+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:49.755 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:49.753+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:50.159 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:50.158+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:50.786 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:50.786+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:51.201 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:51.200+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:51.796 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:51.795+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:52.176 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:52.175+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:52.769 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:52.768+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:53.220 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:53.220+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:53.796 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:53.795+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:54.222 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:54.221+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:54.839 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:54.839+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:55.233 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:55.232+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:55.813 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:55.812+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:56.227 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:56.226+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:56.785 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:56.785+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:57.216 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:57.215+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:57.741 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:57.740+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:58.212 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:58.211+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:58.751 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:58.750+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:59.243 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:53:59.242+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:53:59.754 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:53:59.753+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:00.215 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:00.214+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:00.717 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:00.717+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:01.205 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:01.204+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:01.704 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:01.703+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:02.158 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:02.158+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:02.693 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:02.693+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:03.144 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:03.144+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:03.724 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:03.723+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:04.141 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:04.141+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:04.737 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:04.736+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:05.122 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:05.121+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:05.780 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:05.779+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:06.114 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:06.113+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:06.779 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:06.778+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:07.116 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:07.115+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:07.753 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:07.752+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:08.074 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:08.074+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:08.766 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:08.766+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:09.112 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:09.112+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:09.809 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:09.807+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:10.109 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:10.107+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:10.824 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:10.822+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:11.134 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:11.133+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:11.782 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:11.780+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:12.172 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:12.171+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:12.761 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:12.759+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:13.178 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:13.177+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:13.763 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:13.761+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:14.186 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:14.185+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:14.734 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:14.732+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:15.226 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:15.224+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:15.703 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:15.701+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:16.216 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:16.214+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:16.734 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:16.732+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:17.167 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:17.166+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:17.747 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:17.745+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:18.207 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:18.205+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:18.722 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:18.720+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:19.234 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:19.232+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:19.686 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:19.685+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:20.186 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:20.184+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:20.693 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:20.691+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:21.167 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:21.166+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:21.740 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:21.739+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:22.184 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:22.183+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:22.737 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:22.735+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:23.161 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:23.159+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:23.710 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:23.709+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:24.164 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:24.162+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:24.675 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:24.673+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:25.204 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:25.203+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:25.664 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:25.663+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:26.241 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:26.239+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:26.617 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:26.615+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:27.193 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:27.191+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:27.617 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:27.615+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:28.213 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:28.211+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:28.658 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:28.656+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:29.174 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:29.173+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:29.668 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:29.666+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:30.192 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:30.190+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:30.631 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:30.630+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:31.155 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:31.153+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:31.611 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:31.609+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:32.201 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:32.199+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:32.600 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:32.598+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:33.176 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:33.174+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:33.589 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:33.588+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:34.195 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:34.193+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:34.613 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:34.612+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:35.149 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:35.147+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:35.607 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:35.605+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:36.100 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:36.099+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:36.587 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:36.585+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:37.138 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:37.137+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:37.568 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:37.566+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:38.160 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:38.158+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:38.599 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:38.598+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:39.112 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:39.110+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:39.563 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:39.562+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:40.162 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:40.161+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:40.606 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:40.604+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:41.116 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:41.115+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:41.593 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:41.591+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:42.151 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:42.149+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:42.626 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:42.625+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:43.162 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:43.160+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:43.642 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:43.640+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:44.114 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:44.112+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:44.644 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:44.643+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:45.101 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:45.099+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:45.654 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:45.652+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:46.080 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:46.078+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:46.626 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:46.625+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:47.086 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:47.085+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:47.611 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:47.610+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:48.101 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:48.101+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:48.620 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:48.620+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:49.107 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:49.107+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:49.666 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:49.666+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:50.117 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:50.117+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:50.679 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:50.678+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:51.148 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:51.147+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:51.647 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:51.646+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:52.125 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:52.125+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:52.669 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:52.668+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:53.145 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:53.145+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:53.680 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:53.680+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:54.165 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:54.164+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:54.646 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:54.645+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:55.198 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:55.197+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:55.622 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:55.621+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:56.189 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:56.189+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:56.574 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:56.573+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:57.219 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:57.218+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:57.581 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:57.580+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:58.220 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:58.220+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:58.614 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:58.613+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:59.258 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:54:59.257+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:54:59.658 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:54:59.658+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:00.235 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:00.234+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:00.623 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:00.622+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:01.276 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:01.275+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:01.650 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:01.650+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:02.320 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:02.320+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:02.614 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:02.614+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:03.360 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:03.359+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:03.598 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:03.597+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:04.352 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:04.351+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:04.567 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:04.567+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:05.388 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:05.387+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:05.610 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:05.610+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:06.340 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:06.339+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:06.654 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:06.653+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:07.347 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:07.346+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:07.655 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:07.654+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:08.397 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:08.396+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:08.705 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:08.703+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:09.363 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:09.361+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:09.718 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:09.716+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:10.376 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:10.374+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:10.673 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:10.671+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:11.355 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:11.354+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:11.631 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:11.629+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:12.389 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:12.387+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:12.593 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:12.592+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:13.383 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:13.382+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:13.635 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:13.634+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:14.382 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:14.380+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:14.663 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:14.661+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:15.381 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:15.379+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:15.650 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:15.649+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:16.425 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:16.423+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:16.620 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:16.618+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:17.381 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:17.379+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:17.592 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:17.590+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:18.357 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:18.356+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:18.546 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:18.544+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:19.399 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:19.397+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:19.500 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:19.498+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:20.376 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:20.374+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:20.543 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:20.541+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:21.366 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:21.364+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:21.589 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:21.587+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:22.351 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:22.350+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:22.596 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:22.594+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:23.337 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:23.335+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:23.589 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:23.587+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:24.326 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:24.324+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:24.591 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:24.589+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:25.353 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:25.351+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:25.623 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:25.621+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:26.350 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:26.348+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:26.638 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:26.637+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:27.314 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:27.312+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:27.613 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:27.611+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:28.309 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:28.307+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:28.640 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:28.638+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:29.287 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:29.285+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:29.603 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:29.602+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:30.313 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:30.312+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:30.568 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:30.566+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:31.276 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:31.274+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:31.596 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:31.595+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:32.268 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:32.266+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:32.642 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:32.640+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:33.276 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:33.275+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:33.678 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:33.677+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:34.242 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:34.240+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:34.686 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:34.684+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:35.228 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:35.227+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:35.704 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:35.702+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:36.183 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:36.181+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:36.739 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:36.737+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:37.224 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:37.222+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:37.705 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:37.703+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:38.211 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:38.210+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:38.683 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:38.681+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:39.240 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:39.238+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:39.640 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:39.638+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:40.251 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:40.249+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:40.606 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:40.604+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:41.270 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:41.269+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:41.560 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:41.559+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:42.278 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:42.276+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:42.605 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:42.603+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:43.310 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:43.309+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:43.633 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:43.632+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:44.272 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:44.270+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:44.626 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:44.624+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:45.284 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:45.283+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:45.658 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:45.657+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:46.251 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:46.251+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:46.643 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:46.643+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:47.283 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:47.283+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:47.609 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:47.608+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:48.308 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:48.307+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:48.644 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:48.643+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:49.333 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:49.332+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:49.654 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:49.653+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:50.294 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:50.293+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:50.640 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:50.640+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:51.283 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:51.282+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:51.648 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:51.647+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:52.240 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:52.239+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:52.676 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:52.675+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:53.240 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:53.240+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:53.640 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:53.639+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:54.237 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:54.236+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:54.606 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:54.606+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:55.256 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:55.255+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:55.610 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:55.610+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:56.239 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:56.239+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:56.586 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:56.586+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:57.216 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:57.216+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:57.596 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:57.595+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:58.257 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:58.257+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:58.593 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:58.592+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:59.267 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:55:59.266+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:55:59.633 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:55:59.633+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:00.241 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:00.240+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:00.599 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:00.599+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:01.277 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:01.277+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:01.591 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:01.591+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:02.311 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:02.311+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:02.609 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:02.609+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:03.279 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:03.279+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:03.573 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:03.573+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:04.247 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:04.246+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:04.576 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:04.575+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:05.264 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:05.263+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:05.545 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:05.543+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:06.220 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:06.218+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:06.502 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:06.500+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:07.224 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:07.222+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:07.475 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:07.474+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:08.272 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:08.270+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:08.506 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:08.504+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:09.242 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:09.240+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:09.545 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:09.544+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:10.265 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:10.263+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:10.587 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:10.586+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:11.250 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:11.248+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:11.547 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:11.545+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:12.288 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:12.286+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:12.589 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:12.588+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:13.251 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:13.250+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:13.609 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:13.608+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:14.257 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:14.255+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:14.621 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:14.619+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:15.286 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:15.284+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:15.589 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:15.588+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:16.287 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:16.285+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:16.632 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:16.631+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:17.242 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:17.241+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:17.626 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:17.624+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:18.291 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:18.289+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:18.580 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:18.578+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:19.299 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:19.297+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:19.569 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:19.567+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:20.334 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:20.332+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:20.586 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:20.583+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:21.316 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:21.314+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:21.635 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:21.634+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:22.281 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:22.279+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:22.623 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:22.621+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:23.246 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:23.244+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:23.634 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:23.633+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:24.249 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:24.248+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:24.618 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:24.617+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:25.240 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:25.239+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:25.617 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:25.615+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:26.248 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:26.246+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:26.660 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:26.658+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:27.270 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:27.268+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:27.642 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:27.640+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:28.319 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:28.317+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:28.626 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:28.625+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:29.325 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:29.323+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:29.613 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:29.611+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:30.295 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:30.293+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:30.578 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:30.576+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:31.302 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:31.300+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:31.613 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:31.612+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:32.337 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:32.336+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:32.617 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:32.615+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:33.345 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:33.344+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:33.640 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:33.638+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:34.388 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:34.386+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:34.621 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:34.619+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:35.394 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:35.392+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:35.580 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:35.578+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:36.405 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:36.404+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:36.542 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:36.541+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:37.358 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:37.356+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:37.518 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:37.516+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:38.392 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:38.390+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:38.526 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:38.524+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:39.432 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:39.430+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:39.505 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:39.504+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:40.463 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:40.461+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:40.553 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:40.551+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:41.486 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:41.485+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:41.532 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:41.531+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:42.455 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:42.455+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:42.573 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:42.572+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:43.498 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:43.497+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:43.611 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:43.610+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:44.534 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:44.533+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:44.646 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:44.645+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:45.552 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:45.551+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:45.673 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:45.672+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:46.531 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:46.530+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:46.720 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:46.720+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:47.504 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:47.504+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:47.765 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:47.764+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:48.515 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:48.515+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:48.788 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:48.787+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:49.521 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:49.521+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:49.815 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:49.814+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:50.570 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:50.570+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:50.780 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:50.779+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:51.565 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:51.564+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:51.763 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:51.762+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:52.560 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:52.559+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:52.806 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:52.806+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:53.544 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:53.543+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:53.797 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:53.796+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:54.518 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:54.517+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:54.753 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:54.752+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:55.472 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:55.471+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:55.790 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:55.790+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:56.453 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:56.452+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:56.768 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:56.767+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:57.469 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:57.468+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:57.788 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:57.787+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:58.454 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:58.454+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:58.771 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:58.771+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:59.413 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:56:59.412+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:56:59.806 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:56:59.806+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:00.436 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:00.435+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:00.785 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:00.784+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:01.401 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:01.400+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:01.807 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:01.806+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:02.393 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:02.392+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:02.848 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:02.846+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:03.385 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:03.383+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:03.814 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:03.812+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:04.383 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:04.381+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:04.766 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:04.764+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:05.347 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:05.345+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:05.815 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:05.813+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:06.361 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:06.360+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:06.855 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:06.853+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:07.315 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:07.313+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:07.889 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:07.887+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:08.337 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:08.335+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:08.863 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:08.861+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:09.324 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:09.323+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:09.840 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:09.838+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:10.325 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:10.323+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:10.809 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:10.807+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:11.334 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:11.332+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:11.779 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:11.778+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:12.347 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:12.345+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:12.743 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:12.741+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:13.345 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:13.343+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:13.788 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:13.786+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:14.337 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:14.335+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:14.812 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:14.810+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:15.342 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:15.340+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:15.793 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:15.791+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:16.359 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:16.357+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:16.747 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:16.745+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:17.391 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:17.390+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:17.705 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:17.704+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:18.438 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:18.437+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:18.730 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:18.728+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:19.421 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:19.419+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:19.684 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:19.682+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:20.397 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:20.395+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:20.675 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:20.673+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:21.425 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:21.423+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:21.699 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:21.697+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:22.461 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:22.460+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:22.718 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:22.716+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:23.449 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:23.447+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:23.677 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:23.676+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:24.486 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:24.485+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:24.682 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:24.681+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:25.441 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:25.439+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:25.699 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:25.697+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:26.462 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:26.460+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:26.745 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:26.743+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:27.497 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:27.496+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:27.732 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:27.730+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:28.508 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:28.506+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:28.695 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:28.693+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:29.547 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:29.545+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:29.711 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:29.709+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:30.501 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:30.499+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:30.758 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:30.756+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:31.473 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:31.471+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:31.742 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:31.741+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:32.491 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:32.489+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:32.785 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:32.783+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:33.513 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:33.511+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:33.747 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:33.745+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:34.494 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:34.492+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:34.790 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:34.789+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:35.507 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:35.505+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:35.797 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:35.795+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:36.461 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:36.459+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:36.755 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:36.753+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:37.420 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:37.418+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:37.722 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:37.721+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:38.437 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:38.435+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:38.745 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:38.743+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:39.406 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:39.404+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:39.792 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:39.791+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:40.437 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:40.436+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:40.789 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:40.789+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:41.421 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:41.420+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:41.815 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:41.815+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:42.414 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:42.413+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:42.842 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:42.837+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:43.404 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:43.403+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:43.834 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:43.833+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:44.395 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:44.394+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:44.805 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:44.805+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:45.392 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:45.391+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:45.819 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:45.818+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:46.346 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:46.345+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:46.815 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:46.814+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:47.385 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:47.384+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:47.829 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:47.828+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:48.431 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:48.431+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:48.865 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:48.864+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:49.454 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:49.453+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:49.861 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:49.860+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:50.444 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:50.443+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:50.862 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:50.861+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:51.482 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:51.481+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:51.821 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:51.821+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:52.465 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:52.465+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:52.845 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:52.844+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:53.513 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:53.512+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:53.816 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:53.815+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:54.524 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:54.524+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:54.774 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:54.773+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:55.512 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:55.511+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:55.742 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:55.741+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:56.464 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:56.463+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:56.716 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:56.716+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:57.462 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:57.462+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:57.748 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:57.747+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:58.467 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:58.467+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:58.707 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:58.706+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:59.462 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:57:59.461+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:57:59.686 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:57:59.685+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:00.449 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:00.447+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:00.730 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:00.728+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:01.401 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:01.400+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:01.715 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:01.713+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:02.405 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:02.404+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:02.753 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:02.752+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:03.358 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:03.356+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:03.739 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:03.738+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:04.407 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:04.405+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:04.717 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:04.715+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:05.454 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:05.452+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:05.749 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:05.747+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:06.424 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:06.423+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:06.785 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:06.783+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:07.442 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:07.441+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:07.741 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:07.739+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:08.474 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:08.473+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:08.752 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:08.750+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:09.445 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:09.443+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:09.749 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:09.748+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:10.420 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:10.418+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:10.756 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:10.754+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:11.405 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:11.403+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:11.718 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:11.716+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:12.361 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:12.360+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:12.710 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:12.708+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:13.325 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:13.323+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:13.672 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:13.670+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:14.313 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:14.311+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:14.706 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:14.704+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:15.296 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:15.294+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:15.743 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:15.741+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:16.253 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:16.251+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:16.791 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:16.788+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:17.235 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:17.234+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:17.761 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:17.759+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:18.264 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:18.262+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:18.753 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:18.751+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:19.257 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:19.256+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:19.784 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:19.782+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:20.266 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:20.264+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:20.782 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:20.780+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:21.303 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:21.301+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:21.797 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:21.796+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:22.334 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:22.332+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:22.813 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:22.811+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:23.291 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:23.290+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:23.802 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:23.800+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:24.244 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:24.242+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:24.793 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:24.791+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:25.205 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:25.204+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:25.812 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:25.810+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:26.166 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:26.164+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:26.813 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:26.811+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:27.213 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:27.211+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:27.820 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:27.818+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:28.248 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:28.246+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:28.860 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:28.859+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:29.293 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:29.291+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:29.866 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:29.865+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:30.325 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:30.323+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:30.892 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:30.890+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:31.281 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:31.279+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:31.936 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:31.934+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:32.291 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:32.289+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:32.983 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:32.981+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:33.307 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:33.306+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:34.014 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:34.012+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:34.325 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:34.323+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:35.014 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:35.013+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:35.341 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:35.339+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:36.056 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:36.054+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:36.294 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:36.292+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:37.090 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:37.089+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:37.293 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:37.292+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:38.074 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:38.074+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:38.337 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:38.336+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:39.026 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:39.026+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:39.330 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:39.329+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:40.047 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:40.046+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:40.304 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:40.303+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:41.070 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:41.070+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:41.270 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:41.269+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:42.037 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:42.036+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:42.311 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:42.310+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:43.040 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:43.040+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:43.335 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:43.334+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:44.026 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:44.025+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:44.339 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:44.338+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:45.025 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:45.022+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:45.340 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:45.340+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:46.053 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:46.053+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:46.308 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:46.307+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:47.071 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:47.070+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:47.267 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:47.266+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:48.068 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:48.067+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:48.307 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:48.306+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:49.103 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:49.103+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:49.346 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:49.345+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:50.069 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:50.068+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:50.380 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:50.379+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:51.057 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:51.056+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:51.359 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:51.358+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:52.078 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:52.078+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:52.390 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:52.389+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:53.053 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:53.052+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:53.397 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:53.396+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:54.009 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:54.008+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:54.390 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:54.390+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:55.005 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:55.004+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:55.416 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:55.416+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:55.997 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:55.996+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:56.408 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:56.407+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:57.019 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:57.018+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:57.453 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:57.452+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:58.041 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:58.039+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:58.406 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:58.405+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:59.002 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:58:59.000+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:58:59.386 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:58:59.384+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:00.022 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:00.021+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:00.376 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:00.375+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:01.030 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:01.028+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:01.332 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:01.331+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:02.035 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:02.033+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:02.359 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:02.357+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:02.994 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:02.992+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:03.366 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:03.364+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:03.976 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:03.975+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:04.381 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:04.379+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:04.990 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:04.988+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:05.352 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:05.350+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:05.974 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:05.972+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:06.353 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:06.351+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:06.976 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:06.974+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:07.378 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:07.376+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:07.933 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:07.931+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:08.421 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:08.419+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:08.926 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:08.924+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:09.446 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:09.444+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:09.958 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:09.956+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:10.490 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:10.488+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:10.929 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:10.927+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:11.476 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:11.474+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:11.931 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:11.929+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:12.523 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:12.521+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:12.908 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:12.906+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:13.564 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:13.562+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:13.894 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:13.893+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:14.602 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:14.600+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:14.848 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:14.846+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:15.600 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:15.598+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:15.896 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:15.894+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:16.563 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:16.561+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:16.923 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:16.921+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:17.522 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:17.520+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:17.939 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:17.938+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:18.552 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:18.550+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:18.946 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:18.944+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:19.516 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:19.514+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:19.955 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:19.953+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:20.540 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:20.538+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:21.002 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:21.001+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:21.587 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:21.585+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:21.986 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:21.984+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:22.574 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:22.572+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:23.013 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:23.011+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:23.533 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:23.531+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:23.974 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:23.973+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:24.513 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:24.511+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:25.006 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:25.004+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:25.471 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:25.470+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:25.958 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:25.956+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:26.479 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:26.477+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:26.917 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:26.915+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:27.455 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:27.453+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:27.878 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:27.876+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:28.480 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:28.478+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:28.879 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:28.877+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:29.516 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:29.515+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:29.870 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:29.868+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:30.535 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:30.533+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:30.875 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:30.873+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:31.494 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:31.492+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:31.926 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:31.924+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:32.500 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:32.498+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:32.931 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:32.929+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:33.516 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:33.514+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:33.918 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:33.917+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:34.512 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:34.511+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:34.969 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:34.969+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:35.482 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:35.481+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:35.971 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:35.970+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:36.500 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:36.499+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:36.956 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:36.955+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:37.462 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:37.461+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:37.981 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:37.981+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:38.496 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:38.495+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:38.956 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:38.955+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:39.496 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:39.495+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:39.981 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:39.980+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:40.533 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:40.533+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:40.949 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:40.948+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:41.502 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:41.501+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:41.932 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:41.931+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:42.549 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:42.548+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:42.961 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:42.960+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:43.500 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:43.499+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:43.940 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:43.939+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:44.520 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:44.520+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:44.900 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:44.899+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:45.483 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:45.482+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:45.926 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:45.925+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:46.512 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:46.511+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:46.881 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:46.881+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:47.493 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:47.492+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:47.886 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:47.885+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:48.452 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:48.451+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:48.899 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:48.898+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:49.480 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:49.479+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:49.948 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:49.947+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:50.482 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:50.481+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:50.975 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:50.974+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:51.467 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:51.466+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:51.959 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:51.958+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:52.495 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:52.494+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:52.921 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:52.920+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:53.500 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:53.499+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:53.891 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:53.890+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:54.502 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:54.501+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:54.877 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:54.876+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:55.501 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:55.499+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:55.835 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:55.834+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:56.550 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:56.548+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:56.797 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:56.795+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:57.568 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:57.566+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:57.751 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:57.749+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:58.541 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:58.539+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:58.752 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:58.750+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:59.543 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T22:59:59.541+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T22:59:59.758 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T22:59:59.756+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:00.526 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:00.524+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:00.780 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:00.778+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:01.513 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:01.511+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:01.768 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:01.766+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:02.532 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:02.530+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:02.784 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:02.782+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:03.515 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:03.514+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:03.756 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:03.754+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:04.490 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:04.488+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:04.772 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:04.770+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:05.511 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:05.509+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:05.804 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:05.802+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:06.483 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:06.481+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:06.766 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:06.765+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:07.524 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:07.522+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:07.789 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:07.787+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:08.561 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:08.559+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:08.819 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:08.818+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:09.593 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:09.591+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:09.821 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:09.819+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:10.567 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:10.566+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:10.778 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:10.776+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:11.542 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:11.540+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:11.801 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:11.799+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:12.522 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:12.520+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:12.788 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:12.786+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:13.543 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:13.541+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:13.763 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:13.761+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:14.576 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:14.574+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:14.768 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:14.767+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:15.608 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:15.606+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:15.788 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:15.786+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:16.601 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:16.599+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:16.765 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:16.763+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:17.559 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:17.557+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:17.739 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:17.738+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:18.572 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:18.571+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:18.747 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:18.745+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:19.554 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:19.552+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:19.724 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:19.722+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:20.523 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:20.521+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:20.709 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:20.707+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:21.519 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:21.517+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:21.684 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:21.682+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:22.507 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:22.505+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:22.690 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:22.689+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:23.472 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:23.470+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:23.693 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:23.691+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:24.425 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:24.424+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:24.683 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:24.681+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:25.473 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:25.471+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:25.715 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:25.713+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:26.489 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:26.487+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:26.685 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:26.683+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:27.530 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:27.527+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:27.682 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:27.680+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:28.490 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:28.488+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:28.646 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:28.644+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:29.449 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:29.447+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:29.662 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:29.661+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:30.449 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:30.448+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:30.665 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:30.663+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:31.445 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:31.443+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:31.618 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:31.616+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:32.397 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:32.395+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:32.656 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:32.655+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:33.379 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:33.378+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:33.702 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:33.701+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:34.338 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:34.337+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:34.708 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:34.707+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:35.369 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:35.368+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:35.665 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:35.664+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:36.369 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:36.368+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:36.644 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:36.643+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:37.413 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:37.412+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:37.693 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:37.693+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:38.429 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:38.428+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:38.690 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:38.689+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:39.470 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:39.469+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:39.733 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:39.733+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:40.515 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:40.514+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:40.761 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:40.760+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:41.559 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:41.558+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:41.765 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:41.764+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:42.511 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:42.510+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:42.776 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:42.775+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:43.511 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:43.509+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:43.731 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:43.727+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:44.532 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:44.531+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:44.723 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:44.722+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:45.568 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:45.567+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:45.729 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:45.728+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:46.581 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:46.579+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:46.696 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:46.695+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:47.543 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:47.542+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:47.720 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:47.719+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:48.586 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:48.585+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:48.723 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:48.722+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:49.634 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:49.633+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:49.720 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:49.719+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:50.640 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:50.639+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:50.746 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:50.746+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:51.631 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:51.630+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:51.724 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:51.723+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:52.650 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:52.649+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:52.749 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:52.747+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:53.604 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:53.601+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:53.723 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:53.721+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:54.603 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:54.601+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:54.730 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:54.728+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:55.581 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:55.579+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:55.689 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:55.688+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:56.538 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:56.536+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:56.659 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:56.657+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:57.550 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:57.548+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:57.696 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:57.694+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:58.580 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:58.578+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:58.705 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:58.703+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:59.600 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:00:59.598+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:00:59.681 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:00:59.679+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:00.629 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:00.628+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:00.721 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:00.719+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:01.677 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:01.675+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:01.759 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:01.757+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:02.662 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:02.660+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:02.755 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:02.754+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:03.660 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:03.658+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:03.742 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:03.740+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:04.612 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:04.610+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:04.787 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:04.786+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:05.591 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:05.589+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:05.821 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:05.820+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:06.596 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:06.594+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:06.809 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:06.807+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:07.643 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:07.641+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:07.802 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:07.800+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:08.674 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:08.672+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:08.840 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:08.838+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:09.670 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:09.668+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:09.820 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:09.818+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:10.683 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:10.682+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:10.828 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:10.826+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:11.714 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:11.712+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:11.830 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:11.828+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:12.666 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:12.665+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:12.825 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:12.824+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:13.628 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:13.627+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:13.800 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:13.798+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:14.655 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:14.654+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:14.804 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:14.803+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:15.697 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:15.695+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:15.802 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:15.800+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:16.699 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:16.697+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:16.772 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:16.770+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:17.672 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:17.670+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:17.733 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:17.731+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:18.706 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:18.704+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:18.768 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:18.766+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:19.661 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:19.659+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:19.804 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:19.802+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:20.685 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:20.683+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:20.789 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:20.788+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:21.655 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:21.653+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:21.748 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:21.746+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:22.643 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:22.641+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:22.770 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:22.769+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:23.645 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:23.644+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:23.751 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:23.749+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:24.660 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:24.659+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:24.719 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:24.718+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:25.626 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:25.624+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:25.715 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:25.714+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:26.665 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:26.663+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:26.683 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:26.681+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:27.644 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:27.642+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:27.657 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:27.656+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:28.613 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:28.612+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:28.679 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:28.677+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:29.576 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:29.574+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:29.654 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:29.653+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:30.603 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:30.603+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:30.702 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:30.702+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:31.581 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:31.580+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:31.678 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:31.677+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:32.561 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:32.560+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:32.670 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:32.669+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:33.514 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:33.513+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:33.665 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:33.664+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:34.519 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:34.518+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:34.641 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:34.640+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:35.563 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:35.562+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:35.630 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:35.630+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:36.582 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:36.581+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:36.674 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:36.673+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:37.558 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:37.557+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:37.639 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:37.638+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:38.597 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:38.596+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:38.657 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:38.657+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:39.631 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:39.630+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:39.643 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:39.643+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:40.649 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:40.648+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:40.665 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:40.664+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:41.606 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:41.605+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:41.707 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:41.706+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:42.605 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:42.604+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:42.732 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:42.731+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:43.624 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:43.623+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:43.720 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:43.719+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:44.630 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:44.629+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:44.750 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:44.749+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:45.678 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:45.677+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:45.778 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:45.777+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:46.695 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:46.694+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:46.788 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:46.788+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:47.694 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:47.693+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:47.836 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:47.835+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:48.733 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:48.732+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:48.843 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:48.842+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:49.727 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:49.727+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:49.833 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:49.833+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:50.714 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:50.712+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:50.853 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:50.851+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:51.742 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:51.741+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:51.843 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:51.841+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:52.785 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:52.783+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:52.810 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:52.808+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:53.810 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:53.808+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:53.834 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:53.832+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:54.826 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:54.824+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:54.833 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:54.831+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:55.806 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:55.803+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:55.866 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:55.864+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:56.817 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:56.815+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:56.834 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:56.833+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:57.857 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:57.855+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:57.878 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:57.876+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:58.873 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:58.871+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:58.882 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:58.880+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:59.855 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:01:59.853+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:01:59.899 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:01:59.898+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:00.829 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:00.827+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:00.937 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:00.936+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:01.781 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:01.779+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:01.971 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:01.970+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:02.790 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:02.788+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:02.949 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:02.947+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:03.804 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:03.802+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:03.996 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:03.995+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:04.784 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:04.782+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:05.036 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:05.034+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:05.770 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:05.769+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:05.991 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:05.988+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:06.809 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:06.807+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:07.036 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:07.035+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:07.823 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:07.821+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:08.042 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:08.040+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:08.824 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:08.823+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:09.006 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:09.005+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:09.856 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:09.853+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:10.037 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:10.035+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:10.837 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:10.836+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:11.028 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:11.027+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:11.878 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:11.876+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:12.008 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:12.006+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:12.887 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:12.885+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:12.960 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:12.958+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:13.930 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:13.928+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:13.985 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:13.983+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:14.890 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:14.889+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:14.984 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:14.982+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:15.935 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:15.933+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:15.947 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:15.945+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:16.967 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:16.965+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:16.968 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:16.966+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:17.933 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:17.931+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:17.954 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:17.952+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:18.952 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:18.950+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:18.955 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:18.954+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:19.908 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:19.906+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:19.959 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:19.957+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:20.870 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:20.869+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:20.938 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:20.936+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:21.900 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:21.898+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:21.921 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:21.919+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:22.876 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:22.875+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:22.903 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:22.902+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:23.866 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:23.864+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:23.870 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:23.869+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:24.905 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:24.904+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:24.909 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:24.907+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:25.892 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:25.890+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:25.897 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:25.896+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:26.925 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:26.923+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:26.925 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:26.924+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:27.945 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:27.944+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:27.972 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:27.972+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:28.953 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:28.952+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:28.985 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:28.985+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:29.972 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:29.971+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:29.991 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:29.991+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:30.980 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:30.979+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:31.041 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:31.041+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:31.980 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:31.979+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:32.007 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:32.006+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:32.965 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:32.964+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:33.046 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:33.045+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:33.976 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:33.975+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:34.030 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:34.029+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:34.939 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:34.938+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:35.080 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:35.079+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:35.910 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:35.909+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:36.039 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:36.037+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:36.864 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:36.863+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:37.014 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:37.013+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:37.852 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:37.851+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:38.065 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:38.064+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:38.840 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:38.839+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:39.024 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:39.023+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:39.804 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:39.804+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:40.039 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:40.039+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:40.762 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:40.761+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:41.029 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:41.028+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:41.736 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:41.735+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:42.004 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:42.003+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:42.763 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:42.762+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:42.986 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:42.985+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:43.740 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:43.739+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:43.971 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:43.970+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:44.747 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:44.746+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:45.012 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:45.011+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:45.720 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:45.720+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:46.000 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:45.999+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:46.688 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:46.687+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:46.970 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:46.970+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:47.734 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:47.732+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:47.926 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:47.924+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:48.780 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:48.778+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:48.902 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:48.900+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:49.830 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:49.828+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:49.914 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:49.912+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:50.837 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:50.836+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:50.892 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:50.890+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:51.828 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:51.826+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:51.844 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:51.842+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:52.818 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:52.816+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:52.825 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:52.824+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:53.856 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:53.854+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:53.857 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:53.856+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:54.830 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:54.828+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:54.876 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:54.874+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:55.815 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:55.813+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:55.910 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:55.908+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:56.849 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:56.847+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:56.901 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:56.899+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:57.875 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:57.873+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:57.887 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:57.886+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:58.842 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:58.840+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:58.897 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:58.896+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:59.869 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:02:59.867+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:02:59.898 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:02:59.896+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:00.851 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:00.849+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:00.904 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:00.903+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:01.883 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:01.877+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:01.889 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:01.888+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:02.889 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:02.887+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:02.889 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:02.888+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:03.856 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:03.854+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:03.892 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:03.890+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:04.814 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:04.812+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:04.933 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:04.931+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:05.862 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:05.860+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:05.936 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:05.934+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:06.855 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:06.854+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:06.946 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:06.944+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:07.845 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:07.843+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:07.925 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:07.923+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:08.825 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:08.823+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:08.904 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:08.903+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:09.843 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:09.842+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:09.919 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:09.917+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:10.893 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:10.891+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:10.955 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:10.953+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:11.917 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:11.914+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:11.929 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:11.927+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:12.874 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:12.872+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:12.888 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:12.886+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:13.842 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:13.841+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:13.881 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:13.879+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:14.811 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:14.809+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:14.870 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:14.868+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:15.805 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:15.803+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:15.843 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:15.841+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:16.790 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:16.789+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:16.826 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:16.824+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:17.769 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:17.767+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:17.871 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:17.869+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:18.771 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:18.769+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:18.855 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:18.853+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:19.725 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:19.723+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:19.857 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:19.855+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:20.768 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:20.765+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:20.892 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:20.890+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:21.803 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:21.801+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:21.928 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:21.927+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:22.769 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:22.767+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:22.901 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:22.900+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:23.810 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:23.808+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:23.910 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:23.909+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:24.855 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:24.854+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:24.910 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:24.909+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:25.874 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:25.874+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:25.896 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:25.895+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:26.857 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:26.857+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:26.861 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:26.860+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:27.848 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:27.847+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:27.886 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:27.885+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:28.806 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:28.806+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:28.890 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:28.889+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:29.820 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:29.819+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:29.866 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:29.865+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:30.846 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:30.846+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:30.911 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:30.910+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:31.855 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:31.854+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:31.939 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:31.938+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:32.887 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:32.886+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:32.925 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:32.924+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:33.866 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:33.866+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:33.952 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:33.951+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:34.835 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:34.834+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:34.987 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:34.986+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:35.801 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:35.800+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:35.956 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:35.955+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:36.807 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:36.806+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:36.929 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:36.928+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:37.820 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:37.819+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:37.977 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:37.976+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:38.816 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:38.815+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:38.956 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:38.955+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:39.767 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:39.766+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:39.970 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:39.969+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:40.733 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:40.732+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:40.975 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:40.974+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:41.747 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:41.746+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:42.017 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:42.016+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:42.768 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:42.767+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:43.033 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:43.032+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:43.798 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:43.797+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:44.068 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:44.068+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:44.811 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:44.810+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:45.036 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:45.034+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:45.826 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:45.824+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:46.032 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:46.030+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:46.781 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:46.779+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:47.080 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:47.078+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:47.761 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:47.759+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:48.106 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:48.104+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:48.808 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:48.806+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:49.095 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:49.093+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:49.818 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:49.816+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:50.084 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:50.082+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:50.857 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:50.855+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:51.102 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:51.100+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:51.899 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:51.897+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:52.119 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:52.117+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:52.905 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:52.903+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:53.140 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:53.139+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:53.945 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:53.943+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:54.180 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:54.177+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:54.901 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:54.899+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:55.172 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:55.170+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:55.948 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:55.947+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:56.171 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:56.169+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:56.912 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:56.910+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:57.193 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:57.191+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:57.902 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:57.900+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:58.208 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:58.206+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:58.942 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:58.940+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:59.190 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:03:59.188+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:03:59.952 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:03:59.950+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:00.202 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:00.200+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:00.966 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:00.964+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:01.180 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:01.178+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:01.995 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:01.994+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:02.137 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:02.136+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:03.046 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:03.044+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:03.093 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:03.091+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:04.003 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:04.002+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:04.092 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:04.090+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:04.972 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:04.970+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:05.130 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:05.128+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:05.925 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:05.923+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:06.105 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:06.103+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:06.893 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:06.891+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:07.101 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:07.099+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:07.937 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:07.935+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:08.120 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:08.118+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:08.982 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:08.980+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:09.089 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:09.087+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:09.949 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:09.947+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:10.121 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:10.119+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:10.966 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:10.964+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:11.135 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:11.134+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:11.944 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:11.942+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:12.139 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:12.138+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:12.921 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:12.919+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:13.157 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:13.155+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:13.876 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:13.874+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:14.112 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:14.110+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:14.833 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:14.830+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:15.158 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:15.156+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:15.870 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:15.868+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:16.113 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:16.111+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:16.842 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:16.840+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:17.120 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:17.118+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:17.853 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:17.851+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:18.113 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:18.111+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:18.805 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:18.803+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:19.114 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:19.112+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:19.806 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:19.804+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:20.146 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:20.144+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:20.784 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:20.782+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:21.117 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:21.115+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:21.801 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:21.799+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:22.134 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:22.133+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:22.848 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:22.847+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:23.129 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:23.128+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:23.832 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:23.831+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:24.172 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:24.171+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:24.824 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:24.823+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:25.203 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:25.203+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:25.873 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:25.872+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:26.241 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:26.240+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:26.862 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:26.861+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:27.274 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:27.273+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:27.878 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:27.877+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:28.235 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:28.235+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:28.834 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:28.833+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:29.239 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:29.238+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:29.832 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:29.831+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:30.221 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:30.219+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:30.840 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:30.839+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:31.210 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:31.209+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:31.813 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:31.812+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:32.162 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:32.161+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:32.773 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:32.772+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:33.147 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:33.146+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:33.822 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:33.822+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:34.156 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:34.155+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:34.789 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:34.788+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:35.157 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:35.156+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:35.787 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:35.786+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:36.123 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:36.122+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:36.815 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:36.815+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:37.139 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:37.138+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:37.854 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:37.853+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:38.117 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:38.116+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:38.806 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:38.805+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:39.115 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:39.114+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:39.841 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:39.840+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:40.123 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:40.122+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:40.857 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:40.856+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:41.127 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:41.126+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:41.812 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:41.811+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:42.172 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:42.172+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:42.782 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:42.781+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:43.128 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:43.126+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:43.817 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:43.815+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:44.093 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:44.091+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:44.819 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:44.817+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:45.125 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:45.123+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:45.834 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:45.832+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:46.161 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:46.159+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:46.791 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:46.789+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:47.120 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:47.119+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:47.829 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:47.828+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:48.141 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:48.139+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:48.798 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:48.796+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:49.133 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:49.131+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:49.841 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:49.839+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:50.177 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:50.175+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:50.799 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:50.797+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:51.182 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:51.180+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:51.775 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:51.773+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:52.154 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:52.152+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:52.753 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:52.751+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:53.194 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:53.192+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:53.717 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:53.716+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:54.242 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:54.240+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:54.676 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:54.674+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:55.255 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:55.254+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:55.657 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:55.655+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:56.258 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:56.256+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:56.663 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:56.661+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:57.295 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:57.293+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:57.625 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:57.623+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:58.291 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:58.290+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:58.618 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:58.617+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:59.248 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:04:59.246+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:04:59.633 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:04:59.632+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:00.228 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:00.226+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:00.684 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:00.682+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:01.223 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:01.222+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:01.650 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:01.648+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:02.221 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:02.220+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:02.638 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:02.636+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:03.174 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:03.172+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:03.675 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:03.673+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:04.145 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:04.144+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:04.634 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:04.632+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:05.136 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:05.135+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:05.643 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:05.642+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:06.180 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:06.178+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:06.677 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:06.674+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:07.162 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:07.160+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:07.628 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:07.626+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:08.131 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:08.129+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:08.664 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:08.662+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:09.126 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:09.124+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:09.683 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:09.681+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:10.105 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:10.103+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:10.655 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:10.653+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:11.059 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:11.057+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:11.643 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:11.641+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:12.036 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:12.034+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:12.691 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:12.690+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:13.056 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:13.054+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:13.693 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:13.691+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:14.048 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:14.046+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:14.654 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:14.653+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:15.072 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:15.070+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:15.692 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:15.690+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:16.045 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:16.043+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:16.646 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:16.645+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:17.027 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:17.025+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:17.624 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:17.622+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:18.076 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:18.074+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:18.634 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:18.633+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:19.087 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:19.086+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:19.635 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:19.634+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:20.039 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:20.039+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:20.603 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:20.603+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:21.090 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:21.089+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:21.576 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:21.576+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:22.057 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:22.056+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:22.617 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:22.616+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:23.080 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:23.079+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:23.656 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:23.655+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:24.033 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:24.032+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:24.622 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:24.621+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:25.043 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:25.042+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:25.636 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:25.635+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:26.005 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:26.004+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:26.665 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:26.663+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:27.001 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:27.000+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:27.676 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:27.675+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:27.999 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:27.998+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:28.663 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:28.662+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:29.011 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:29.010+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:29.618 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:29.617+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:29.982 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:29.981+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:30.601 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:30.600+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:30.978 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:30.978+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:31.617 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:31.616+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:32.027 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:32.027+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:32.666 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:32.665+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:33.042 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:33.041+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:33.622 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:33.621+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:34.054 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:34.053+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:34.644 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:34.643+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:35.095 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:35.094+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:35.653 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:35.652+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:36.080 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:36.079+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:36.678 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:36.677+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:37.086 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:37.085+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:37.714 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:37.714+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:38.120 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:38.118+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:38.729 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:38.728+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:39.164 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:39.163+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:39.686 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:39.685+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:40.163 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:40.161+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:40.697 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:40.696+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:41.146 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:41.144+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:41.685 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:41.683+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:42.185 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:42.183+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:42.698 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:42.696+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:43.235 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:43.233+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:43.674 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:43.672+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:44.277 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:44.275+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:44.630 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:44.628+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:45.301 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:45.299+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:45.652 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:45.650+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:46.323 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:46.321+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:46.662 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:46.660+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:47.368 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:47.366+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:47.699 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:47.697+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:48.377 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:48.375+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:48.706 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:48.704+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:49.349 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:49.347+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:49.730 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:49.729+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:50.347 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:50.345+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:50.690 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:50.688+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:51.326 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:51.324+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:51.677 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:51.675+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:52.368 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:52.366+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:52.692 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:52.690+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:53.394 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:53.393+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:53.650 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:53.649+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:54.424 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:54.423+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:54.604 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:54.603+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:55.392 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:55.390+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:55.586 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:55.584+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:56.367 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:56.365+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:56.631 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:56.629+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:57.394 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:57.392+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:57.614 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:57.612+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:58.418 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:58.416+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:58.597 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:58.595+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:59.429 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:05:59.427+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:05:59.624 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:05:59.622+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:00.445 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:00.443+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:00.579 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:00.577+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:01.461 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:01.459+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:01.558 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:01.557+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:02.455 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:02.453+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:02.606 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:02.605+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:03.494 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:03.492+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:03.648 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:03.647+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:04.455 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:04.453+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:04.619 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:04.617+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:05.451 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:05.449+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:05.585 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:05.583+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:06.445 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:06.443+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:06.548 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:06.546+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:07.454 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:07.452+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:07.592 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:07.590+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:08.499 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:08.497+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:08.629 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:08.627+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:09.454 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:09.452+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:09.587 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:09.585+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:10.423 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:10.421+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:10.621 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:10.620+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:11.393 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:11.391+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:11.639 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:11.637+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:12.381 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:12.379+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:12.601 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:12.600+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:13.359 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:13.357+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:13.645 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:13.643+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:14.324 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:14.323+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:14.624 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:14.623+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:15.329 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:15.327+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:15.634 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:15.632+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:16.325 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:16.323+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:16.670 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:16.668+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:17.303 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:17.301+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:17.698 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:17.697+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:18.261 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:18.258+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:18.676 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:18.675+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:19.258 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:19.257+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:19.721 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:19.720+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:20.268 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:20.267+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:20.697 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:20.696+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:21.300 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:21.299+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:21.657 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:21.656+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:22.330 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:22.329+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:22.682 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:22.681+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:23.298 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:23.297+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:23.641 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:23.641+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:24.317 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:24.317+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:24.647 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:24.646+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:25.336 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:25.335+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:25.664 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:25.663+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:26.370 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:26.369+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:26.641 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:26.640+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:27.356 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:27.355+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:27.594 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:27.594+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:28.378 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:28.377+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:28.557 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:28.556+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:29.411 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:29.409+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:29.595 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:29.594+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:30.368 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:30.367+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:30.621 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:30.620+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:31.337 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:31.336+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:31.600 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:31.599+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:32.361 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:32.360+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:32.564 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:32.563+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:33.386 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:33.385+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:33.550 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:33.549+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:34.338 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:34.337+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:34.569 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:34.569+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:35.335 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:35.334+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:35.526 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:35.525+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:36.326 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:36.325+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:36.522 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:36.521+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:37.375 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:37.374+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:37.509 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:37.507+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:38.383 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:38.381+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:38.493 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:38.491+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:39.370 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:39.368+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:39.511 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:39.509+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:40.415 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:40.413+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:40.467 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:40.466+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:41.408 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:41.406+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:41.510 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:41.509+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:42.421 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:42.420+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:42.536 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:42.534+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:43.460 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:43.458+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:43.554 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:43.552+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:44.425 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:44.423+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:44.528 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:44.526+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:45.457 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:45.455+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:45.530 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:45.528+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:46.445 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:46.443+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:46.563 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:46.561+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:47.402 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:47.400+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:47.608 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:47.606+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:48.450 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:48.448+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:48.614 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:48.612+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:49.501 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:49.499+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:49.619 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:49.617+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:50.476 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:50.474+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:50.601 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:50.599+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:51.485 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:51.483+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:51.633 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:51.632+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:52.504 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:52.502+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:52.684 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:52.682+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:53.504 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:53.502+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:53.723 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:53.721+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:54.484 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:54.482+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:54.756 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:54.754+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:55.437 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:55.435+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:55.773 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:55.771+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:56.481 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:56.479+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:56.786 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:56.784+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:57.501 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:57.500+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:57.771 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:57.769+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:58.464 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:58.462+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:58.748 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:58.746+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:59.429 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:06:59.427+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:06:59.715 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:06:59.713+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:00.405 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:00.403+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:00.666 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:00.664+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:01.436 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:01.434+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:01.667 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:01.665+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:02.414 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:02.412+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:02.659 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:02.657+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:03.451 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:03.449+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:03.708 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:03.706+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:04.473 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:04.471+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:04.666 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:04.664+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:05.514 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:05.512+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:05.631 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:05.630+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:06.471 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:06.469+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:06.652 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:06.650+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:07.510 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:07.508+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:07.613 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:07.611+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:08.522 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:08.520+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:08.614 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:08.612+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:09.502 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:09.500+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:09.658 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:09.656+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:10.465 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:10.464+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:10.646 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:10.644+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:11.446 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:11.444+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:11.643 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:11.642+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:12.424 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:12.422+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:12.683 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:12.681+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:13.448 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:13.446+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:13.645 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:13.643+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:14.431 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:14.429+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:14.640 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:14.639+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:15.478 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:15.478+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:15.601 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:15.601+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:16.476 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:16.475+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:16.650 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:16.649+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:17.481 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:17.480+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:17.700 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:17.699+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:18.494 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:18.493+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:18.725 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:18.725+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:19.451 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:19.450+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:19.775 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:19.774+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:20.444 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:20.443+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:20.799 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:20.798+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:21.432 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:21.431+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:21.823 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:21.822+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:22.478 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:22.477+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:22.802 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:22.802+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:23.435 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:23.434+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:23.796 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:23.795+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:24.409 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:24.408+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:24.831 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:24.830+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:25.393 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:25.392+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:25.840 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:25.839+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:26.376 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:26.375+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:26.887 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:26.887+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:27.340 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:27.339+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:27.930 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:27.930+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:28.342 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:28.341+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:28.976 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:28.976+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:29.351 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:29.350+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:30.004 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:30.003+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:30.345 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:30.344+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:31.033 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:31.032+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:31.306 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:31.305+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:31.994 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:31.993+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:32.322 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:32.321+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:32.964 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:32.963+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:33.286 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:33.285+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:33.948 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:33.947+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:34.268 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:34.267+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:34.932 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:34.931+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:35.272 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:35.270+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:35.966 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:35.965+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:36.270 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:36.268+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:36.980 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:36.979+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:37.302 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:37.300+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:37.969 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:37.967+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:38.262 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:38.260+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:38.975 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:38.973+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:39.252 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:39.250+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:40.018 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:40.016+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:40.302 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:40.300+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:41.033 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:41.031+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:41.304 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:41.302+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:42.079 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:42.078+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:42.340 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:42.339+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:43.039 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:43.037+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:43.386 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:43.384+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:43.993 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:43.991+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:44.403 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:44.401+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:44.977 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:44.976+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:45.373 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:45.371+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:45.981 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:45.979+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:46.388 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:46.386+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:46.980 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:46.978+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:47.371 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:47.369+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:47.975 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:47.973+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:48.334 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:48.332+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:48.985 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:48.983+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:49.350 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:49.348+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:49.992 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:49.990+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:50.386 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:50.384+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:51.011 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:51.009+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:51.432 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:51.430+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:52.028 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:52.026+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:52.482 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:52.480+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:52.998 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:52.996+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:53.495 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:53.493+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:54.016 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:54.014+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:54.455 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:54.453+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:55.003 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:55.001+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:55.447 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:55.445+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:56.009 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:56.007+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:56.464 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:56.462+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:57.047 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:57.045+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:57.506 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:57.504+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:58.073 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:58.071+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:58.481 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:58.479+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:59.043 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:07:59.041+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:07:59.456 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:07:59.454+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:00.054 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:00.052+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:00.505 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:00.503+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:01.076 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:01.074+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:01.476 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:01.474+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:02.125 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:02.123+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:02.485 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:02.483+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:03.111 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:03.110+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:03.523 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:03.521+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:04.143 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:04.141+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:04.536 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:04.534+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:05.114 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:05.112+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:05.548 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:05.546+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:06.132 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:06.130+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:06.538 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:06.536+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:07.086 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:07.084+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:07.584 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:07.582+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:08.047 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:08.045+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:08.598 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:08.596+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:09.002 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:09.000+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:09.611 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:09.609+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:10.041 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:10.039+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:10.646 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:10.644+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:11.087 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:11.085+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:11.624 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:11.623+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:12.073 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:12.071+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:12.593 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:12.591+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:13.081 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:13.080+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:13.597 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:13.596+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:14.102 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:14.101+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:14.646 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:14.645+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:15.108 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:15.107+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:15.690 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:15.689+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:16.131 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:16.130+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:16.728 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:16.727+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:17.137 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:17.136+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:17.705 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:17.704+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:18.175 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:18.174+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:18.745 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:18.743+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:19.164 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:19.163+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:19.779 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:19.778+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:20.152 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:20.151+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:20.744 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:20.743+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:21.178 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:21.177+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:21.759 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:21.758+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:22.212 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:22.211+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:22.797 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:22.796+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:23.164 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:23.163+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:23.825 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:23.824+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:24.168 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:24.167+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:24.857 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:24.856+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:25.160 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:25.160+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:25.836 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:25.835+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:26.205 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:26.204+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:26.844 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:26.843+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:27.190 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:27.189+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:27.802 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:27.801+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:28.160 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:28.159+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:28.754 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:28.753+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:29.206 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:29.205+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:29.758 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:29.757+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:30.215 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:30.214+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:30.765 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:30.709+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:31.249 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:31.248+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:31.724 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:31.723+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:32.233 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:32.232+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:32.745 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:32.744+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:33.220 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:33.218+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:33.737 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:33.736+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:34.248 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:34.246+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:34.725 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:34.723+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:35.232 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:35.230+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:35.725 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:35.723+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:36.240 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:36.238+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:36.708 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:36.706+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:37.252 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:37.250+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:37.662 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:37.660+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:38.292 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:38.290+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:38.696 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:38.694+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:39.303 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:39.301+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:39.686 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:39.684+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:40.305 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:40.303+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:40.666 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:40.664+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:41.342 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:41.340+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:41.632 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:41.630+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:42.382 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:42.381+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:42.593 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:42.592+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:43.376 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:43.374+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:43.628 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:43.626+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:44.327 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:44.325+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:44.640 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:44.638+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:45.326 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:45.324+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:45.684 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:45.682+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:46.367 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:46.365+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:46.648 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:46.646+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:47.405 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:47.403+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:47.692 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:47.690+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:48.360 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:48.358+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:48.738 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:48.736+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:49.312 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:49.310+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:49.757 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:49.755+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:50.362 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:50.361+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:50.778 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:50.776+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:51.348 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:51.346+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:51.807 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:51.805+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:52.390 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:52.388+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:52.836 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:52.834+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:53.372 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:53.370+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:53.823 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:53.821+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:54.366 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:54.364+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:54.799 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:54.798+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:55.396 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:55.394+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:55.845 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:55.843+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:56.397 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:56.395+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:56.876 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:56.874+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:57.370 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:57.368+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:57.887 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:57.886+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:58.341 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:58.339+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:58.936 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:58.934+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:59.377 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:08:59.375+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:08:59.941 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:08:59.939+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:00.350 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:00.348+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:00.961 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:00.960+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:01.358 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:01.356+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:01.969 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:01.967+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:02.341 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:02.339+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:02.924 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:02.922+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:03.376 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:03.374+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:03.953 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:03.951+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:04.345 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:04.343+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:04.970 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:04.968+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:05.330 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:05.328+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:05.934 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:05.932+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:06.317 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:06.314+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:06.921 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:06.919+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:07.282 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:07.281+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:07.943 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:07.941+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:08.268 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:08.266+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:08.980 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:08.979+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:09.239 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:09.237+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:10.031 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:10.029+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:10.250 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:10.248+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:11.029 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:11.028+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:11.233 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:11.232+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:12.071 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:12.070+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:12.273 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:12.272+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:13.118 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:13.117+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:13.323 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:13.322+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:14.087 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:14.086+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:14.315 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:14.314+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:15.136 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:15.135+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:15.310 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:15.309+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:16.169 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:16.168+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:16.297 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:16.296+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:17.139 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:17.138+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:17.292 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:17.291+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:18.141 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:18.140+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:18.317 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:18.316+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:19.145 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:19.144+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:19.335 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:19.334+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:20.126 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:20.125+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:20.331 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:20.331+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:21.136 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:21.135+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:21.346 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:21.345+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:22.114 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:22.112+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:22.352 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:22.351+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:23.110 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:23.109+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:23.313 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:23.311+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:24.150 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:24.149+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:24.284 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:24.283+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:25.156 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:25.155+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:25.318 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:25.317+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:26.161 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:26.160+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:26.337 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:26.336+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:27.172 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:27.171+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:27.291 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:27.290+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:28.207 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:28.206+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:28.271 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:28.270+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:29.255 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:29.254+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:29.296 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:29.295+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:30.234 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:30.232+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:30.297 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:30.295+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:31.224 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:31.222+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:31.347 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:31.346+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:32.213 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:32.211+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:32.307 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:32.305+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:33.256 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:33.254+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:33.264 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:33.262+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:34.231 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:34.229+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:34.266 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:34.264+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:35.189 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:35.187+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:35.254 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:35.252+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:36.224 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:36.222+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:36.263 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:36.261+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:37.220 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:37.218+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:37.240 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:37.238+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:38.205 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:38.203+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:38.219 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:38.217+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:39.192 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:39.190+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:39.252 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:39.250+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:40.190 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:40.188+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:40.258 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:40.256+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:41.227 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:41.225+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:41.290 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:41.288+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:42.190 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:42.188+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:42.279 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:42.277+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:43.193 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:43.191+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:43.292 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:43.290+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:44.213 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:44.211+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:44.328 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:44.326+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:45.225 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:45.223+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:45.285 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:45.283+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:46.178 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:46.176+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:46.246 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:46.245+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:47.160 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:47.158+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:47.291 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:47.289+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:48.163 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:48.161+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:48.284 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:48.282+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:49.169 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:49.167+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:49.322 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:49.320+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:50.140 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:50.138+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:50.301 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:50.299+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:51.158 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:51.156+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:51.286 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:51.284+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:52.141 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:52.139+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:52.307 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:52.305+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:53.147 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:53.145+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:53.326 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:53.324+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:54.189 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:54.187+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:54.322 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:54.320+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:55.184 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:55.182+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:55.354 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:55.352+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:56.210 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:56.208+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:56.391 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:56.390+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:57.165 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:57.163+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:57.397 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:57.396+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:58.189 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:58.187+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:58.409 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:58.408+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:59.170 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:09:59.168+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:09:59.399 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:09:59.397+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:00.195 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:00.194+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:00.355 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:00.353+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:01.153 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:01.151+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:01.401 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:01.399+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:02.168 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:02.166+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:02.407 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:02.405+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:03.138 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:03.136+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:03.430 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:03.428+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:04.144 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:04.142+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:04.427 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:04.426+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:05.113 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:05.111+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:05.386 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:05.384+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:06.154 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:06.152+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:06.435 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:06.434+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:07.172 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:07.171+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:07.406 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:07.405+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:08.194 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:08.193+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:08.408 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:08.407+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:09.215 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:09.214+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:09.454 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:09.453+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:10.256 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:10.255+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:10.466 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:10.465+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:11.286 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:11.285+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:11.515 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:11.514+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:12.276 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:12.275+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:12.473 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:12.472+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:13.293 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:13.292+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:13.440 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:13.439+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:14.343 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:14.342+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:14.425 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:14.425+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:15.294 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:15.293+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:15.454 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:15.453+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:16.321 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:16.320+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:16.438 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:16.437+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:17.280 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:17.279+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:17.408 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:17.407+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:18.306 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:18.305+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:18.406 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:18.405+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:19.336 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:19.335+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:19.391 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:19.390+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:20.303 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:20.302+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:20.352 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:20.351+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:21.310 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:21.309+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:21.331 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:21.330+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:22.308 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:22.307+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:22.335 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:22.334+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:23.261 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:23.260+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:23.369 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:23.368+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:24.212 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:24.211+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:24.333 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:24.332+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:25.202 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:25.201+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:25.316 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:25.315+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:26.223 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:26.222+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:26.328 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:26.327+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:27.232 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:27.231+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:27.358 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:27.357+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:28.199 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:28.197+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:28.328 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:28.326+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:29.249 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:29.247+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:29.297 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:29.295+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:30.277 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:30.275+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:30.290 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:30.288+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:31.273 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:31.271+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:31.320 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:31.318+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:32.287 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:32.285+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:32.320 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:32.318+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:33.241 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:33.239+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:33.272 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:33.270+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:34.206 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:34.204+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:34.243 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:34.241+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:35.198 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:35.196+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:35.245 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:35.243+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:36.159 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:36.158+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:36.281 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:36.279+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:37.137 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:37.135+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:37.301 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:37.299+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:38.133 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:38.131+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:38.318 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:38.316+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:39.085 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:39.083+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:39.309 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:39.308+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:40.122 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:40.120+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:40.333 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:40.331+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:41.157 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:41.155+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:41.362 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:41.360+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:42.176 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:42.174+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:42.346 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:42.344+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:43.180 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:43.178+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:43.316 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:43.314+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:44.191 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:44.189+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:44.281 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:44.279+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:45.163 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:45.161+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:45.328 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:45.326+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:46.186 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:46.184+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:46.351 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:46.349+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:47.190 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:47.188+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:47.330 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:47.328+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:48.198 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:48.197+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:48.378 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:48.376+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:49.157 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:49.155+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:49.379 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:49.377+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:50.128 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:50.127+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:50.355 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:50.354+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:51.168 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:51.166+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:51.358 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:51.357+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:52.142 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:52.140+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:52.394 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:52.392+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:53.126 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:53.124+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:53.401 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:53.399+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:54.170 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:54.168+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:54.444 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:54.442+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:55.136 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:55.134+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:55.472 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:55.470+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:56.164 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:56.161+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:56.492 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:56.490+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:57.133 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:57.131+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:57.492 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:57.490+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:58.120 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:58.118+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:58.532 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:58.531+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:59.166 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:10:59.164+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:10:59.498 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:10:59.496+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:00.164 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:00.162+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:00.529 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:00.527+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:01.143 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:01.142+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:01.511 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:01.509+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:02.108 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:02.106+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:02.463 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:02.462+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:03.158 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:03.156+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:03.504 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:03.502+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:04.203 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:04.202+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:04.503 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:04.501+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:05.181 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:05.179+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:05.489 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:05.488+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:06.179 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:06.179+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:06.533 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:06.532+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:07.158 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:07.157+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:07.547 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:07.546+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:08.134 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:08.133+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:08.587 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:08.587+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:09.145 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:09.145+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:09.586 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:09.585+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:10.153 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:10.153+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:10.604 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:10.603+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:11.134 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:11.133+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:11.609 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:11.608+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:12.102 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:12.101+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:12.645 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:12.644+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:13.085 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:13.084+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:13.622 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:13.621+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:14.045 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:14.045+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:14.651 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:14.651+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:15.096 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:15.095+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:15.641 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:15.641+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:16.072 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:16.071+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:16.650 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:16.649+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:17.045 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:17.044+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:17.649 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:17.648+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:18.005 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:18.004+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:18.653 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:18.652+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:18.985 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:18.984+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:19.690 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:19.688+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:20.029 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:20.028+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:20.718 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:20.717+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:21.018 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:21.017+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:21.762 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:21.761+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:21.973 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:21.972+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:22.751 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:22.750+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:22.995 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:22.994+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:23.768 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:23.767+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:24.007 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:24.006+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:24.757 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:24.757+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:24.995 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:24.994+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:25.765 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:25.763+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:26.022 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:26.020+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:26.738 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:26.736+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:27.042 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:27.040+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:27.742 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:27.740+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:28.039 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:28.037+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:28.782 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:28.780+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:29.040 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:29.038+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:29.782 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:29.780+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:30.070 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:30.068+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:30.831 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:30.829+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:31.033 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:31.031+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:31.800 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:31.798+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:32.003 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:32.001+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:32.758 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:32.756+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:33.049 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:33.047+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:33.761 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:33.759+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:34.020 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:34.019+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:34.742 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:34.739+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:34.990 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:34.988+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:35.703 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:35.702+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:35.949 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:35.947+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:36.732 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:36.730+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:36.911 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:36.910+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:37.737 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:37.735+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:37.960 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:37.958+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:38.698 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:38.696+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:38.967 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:38.965+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:39.668 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:39.666+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:39.943 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:39.941+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:40.716 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:40.714+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:40.966 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:40.964+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:41.678 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:41.676+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:42.012 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:42.010+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:42.695 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:42.693+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:43.021 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:43.019+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:43.674 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:43.672+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:44.022 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:44.020+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:44.658 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:44.656+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:45.032 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:45.030+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:45.614 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:45.613+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:45.988 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:45.986+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:46.598 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:46.596+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:47.026 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:47.024+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:47.647 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:47.645+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:48.025 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:48.023+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:48.661 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:48.659+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:48.997 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:48.995+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:49.628 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:49.626+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:50.043 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:50.041+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:50.612 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:50.610+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:51.036 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:51.034+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:51.615 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:51.613+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:51.997 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:51.995+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:52.580 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:52.578+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:52.993 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:52.992+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:53.535 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:53.533+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:53.961 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:53.959+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:54.583 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:54.581+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:54.966 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:54.964+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:55.561 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:55.559+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:55.984 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:55.982+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:56.532 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:56.530+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:56.945 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:56.943+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:57.505 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:57.503+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:57.993 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:57.990+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:58.545 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:58.543+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:58.953 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:58.950+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:59.553 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:11:59.551+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:11:59.923 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:11:59.921+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:00.518 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:00.516+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:00.973 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:00.969+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:01.551 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:01.549+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:01.949 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:01.947+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:02.579 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:02.577+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:02.943 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:02.942+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:03.548 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:03.547+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:03.963 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:03.962+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:04.580 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:04.580+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:05.003 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:05.002+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:05.544 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:05.544+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:05.958 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:05.958+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:06.566 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:06.565+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:06.986 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:06.986+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:07.536 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:07.535+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:07.980 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:07.979+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:08.581 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:08.580+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:08.949 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:08.948+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:09.622 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:09.621+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:09.917 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:09.916+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:10.659 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:10.659+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:10.904 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:10.903+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:11.664 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:11.658+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:11.903 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:11.902+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:12.675 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:12.674+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:12.880 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:12.880+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:13.670 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:13.669+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:13.896 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:13.895+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:14.697 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:14.696+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:14.922 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:14.921+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:15.704 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:15.703+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:15.924 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:15.923+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:16.699 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:16.697+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:16.944 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:16.943+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:17.666 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:17.665+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:17.967 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:17.966+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:18.629 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:18.628+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:19.010 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:19.009+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:19.660 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:19.659+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:19.978 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:19.977+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:20.652 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:20.652+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:20.999 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:20.998+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:21.701 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:21.699+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:21.998 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:21.997+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:22.699 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:22.698+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:22.971 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:22.969+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:23.653 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:23.651+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:23.965 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:23.964+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:24.652 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:24.650+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:24.955 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:24.953+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:25.658 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:25.657+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:26.001 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:25.999+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:26.635 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:26.633+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:27.007 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:27.005+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:27.642 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:27.640+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:27.966 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:27.964+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:28.679 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:28.677+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:28.963 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:28.961+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:29.681 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:29.679+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:29.968 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:29.966+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:30.674 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:30.672+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:30.992 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:30.990+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:31.704 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:31.703+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:31.985 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:31.983+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:32.725 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:32.723+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:33.009 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:33.007+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:33.729 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:33.727+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:34.059 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:34.057+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:34.745 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:34.741+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:35.047 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:35.045+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:35.761 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:35.759+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:36.068 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:36.067+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:36.727 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:36.725+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:37.056 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:37.054+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:37.687 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:37.685+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:38.050 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:38.048+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:38.731 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:38.729+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:39.006 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:39.004+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:39.704 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:39.703+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:39.986 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:39.984+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:40.719 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:40.717+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:40.993 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:40.991+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:41.754 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:41.751+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:41.995 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:41.993+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:42.738 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:42.736+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:42.976 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:42.974+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:43.722 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:43.720+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:44.001 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:43.999+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:44.754 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:44.752+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:45.030 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:45.029+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:45.715 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:45.713+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:46.073 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:46.071+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:46.679 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:46.677+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:47.123 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:47.121+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:47.675 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:47.673+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:48.109 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:48.107+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:48.671 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:48.669+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:49.065 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:49.063+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:49.637 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:49.635+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:50.059 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:50.057+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:50.634 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:50.632+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:51.033 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:51.031+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:51.664 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:51.662+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:52.053 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:52.051+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:52.659 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:52.657+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:53.058 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:53.056+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:53.692 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:53.690+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:54.026 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:54.024+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:54.724 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:54.722+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:55.061 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:55.059+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:55.765 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:55.763+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:56.013 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:56.011+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:56.749 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:56.747+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:56.987 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:56.985+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:57.785 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:57.783+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:58.029 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:58.027+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:58.746 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:58.745+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:59.013 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:12:59.012+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:12:59.727 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:12:59.726+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:00.037 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:00.035+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:00.716 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:00.715+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:01.081 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:01.080+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:01.742 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:01.741+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:02.045 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:02.044+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:02.770 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:02.769+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:03.063 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:03.062+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:03.806 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:03.805+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:04.080 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:04.079+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:04.759 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:04.758+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:05.090 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:05.089+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:05.794 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:05.793+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:06.107 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:06.106+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:06.799 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:06.798+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:07.148 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:07.147+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:07.809 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:07.808+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:08.157 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:08.156+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:08.847 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:08.846+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:09.127 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:09.126+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:09.826 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:09.825+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:10.138 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:10.137+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:10.795 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:10.794+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:11.175 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:11.174+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:11.794 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:11.793+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:12.215 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:12.214+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:12.841 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:12.840+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:13.254 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:13.253+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:13.872 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:13.871+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:14.301 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:14.300+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:14.886 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:14.885+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:15.264 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:15.263+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:15.848 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:15.847+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:16.309 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:16.308+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:16.801 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:16.800+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:17.272 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:17.271+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:17.831 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:17.830+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:18.275 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:18.274+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:18.843 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:18.842+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:19.316 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:19.316+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:19.844 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:19.843+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:20.268 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:20.267+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:20.884 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:20.882+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:21.253 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:21.251+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:21.927 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:21.925+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:22.283 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:22.281+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:22.918 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:22.916+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:23.254 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:23.252+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:23.937 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:23.935+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:24.258 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:24.256+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:24.977 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:24.975+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:25.295 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:25.293+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:25.985 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:25.982+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:26.343 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:26.341+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:27.021 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:27.019+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:27.350 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:27.349+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:27.976 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:27.974+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:28.320 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:28.318+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:28.936 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:28.934+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:29.274 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:29.272+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:29.937 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:29.935+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:30.234 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:30.233+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:30.953 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:30.951+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:31.286 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:31.281+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:31.937 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:31.935+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:32.251 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:32.249+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:32.959 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:32.957+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:33.254 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:33.252+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:33.929 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:33.927+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:34.211 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:34.209+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:34.942 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:34.940+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:35.238 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:35.236+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:35.908 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:35.906+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:36.286 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:36.285+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:36.914 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:36.912+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:37.287 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:37.285+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:37.935 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:37.933+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:38.315 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:38.313+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:38.912 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:38.910+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:39.351 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:39.349+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:39.934 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:39.932+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:40.339 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:40.337+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:40.938 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:40.936+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:41.341 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:41.337+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:41.967 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:41.966+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:42.310 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:42.308+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:42.967 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:42.965+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:43.273 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:43.271+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:44.010 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:44.009+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:44.244 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:44.242+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:44.982 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:44.980+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:45.283 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:45.281+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:45.965 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:45.963+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:46.280 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:46.278+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:46.981 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:46.979+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:47.235 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:47.233+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:47.965 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:47.963+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:48.196 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:48.195+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:48.946 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:48.944+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:49.174 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:49.172+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:49.991 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:49.989+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:50.125 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:50.123+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:51.029 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:51.027+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:51.164 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:51.162+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:52.026 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:52.024+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:52.150 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:52.149+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:53.047 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:53.045+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:53.106 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:53.104+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:54.000 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:53.999+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:54.072 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:54.070+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:55.001 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:54.999+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:55.048 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:55.046+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:55.994 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:55.993+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:56.026 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:56.024+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:57.037 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:57.035+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:57.043 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:57.041+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:57.988 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:57.986+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:58.013 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:58.011+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:58.959 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:58.958+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:59.030 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:59.029+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:59.973 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:13:59.972+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:13:59.991 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:13:59.990+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:01.010 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:01.008+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:01.026 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:01.025+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:01.989 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:01.988+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:02.034 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:02.033+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:02.995 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:02.994+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:02.997 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:02.996+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:03.981 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:03.980+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:04.037 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:04.036+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:04.979 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:04.978+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:05.006 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:05.006+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:05.961 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:05.960+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:05.963 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:05.962+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:06.965 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:06.964+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:06.973 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:06.972+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:07.983 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:07.982+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:07.990 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:07.990+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:08.943 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:08.942+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:08.986 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:08.985+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:09.913 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:09.912+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:10.033 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:10.032+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:10.941 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:10.940+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:11.068 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:11.067+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:11.948 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:11.948+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:12.079 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:12.078+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:12.913 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:12.912+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:13.049 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:13.048+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:13.898 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:13.897+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:14.057 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:14.056+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:14.879 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:14.878+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:15.099 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:15.098+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:15.889 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:15.888+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:16.095 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:16.094+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:16.862 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:16.861+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:17.071 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:17.070+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:17.857 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:17.856+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:18.031 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:18.030+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:18.809 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:18.807+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:19.033 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:19.031+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:19.789 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:19.787+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:20.076 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:20.074+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:20.830 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:20.828+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:21.105 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:21.103+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:21.809 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:21.807+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:22.071 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:22.069+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:22.844 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:22.843+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:23.069 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:23.067+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:23.799 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:23.797+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:24.093 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:24.091+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:24.807 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:24.805+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:25.092 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:25.090+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:25.828 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:25.827+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:26.109 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:26.107+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:26.784 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:26.782+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:27.131 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:27.129+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:27.829 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:27.827+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:28.100 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:28.098+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:28.793 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:28.791+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:29.149 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:29.147+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:29.801 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:29.799+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:30.166 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:30.165+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:30.782 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:30.780+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:31.133 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:31.129+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:31.751 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:31.749+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:32.179 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:32.177+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:32.730 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:32.728+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:33.195 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:33.193+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:33.688 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:33.686+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:34.191 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:34.189+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:34.712 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:34.710+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:35.185 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:35.183+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:35.746 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:35.744+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:36.143 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:36.141+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:36.707 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:36.705+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:37.159 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:37.157+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:37.735 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:37.733+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:38.179 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:38.177+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:38.753 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:38.750+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:39.177 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:39.175+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:39.792 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:39.790+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:40.193 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:40.191+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:40.790 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:40.788+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:41.151 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:41.149+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:41.773 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:41.771+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:42.172 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:42.170+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:42.824 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:42.822+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:43.137 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:43.135+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:43.852 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:43.850+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:44.145 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:44.143+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:44.873 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:44.871+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:45.108 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:45.106+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:45.828 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:45.826+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:46.061 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:46.059+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:46.815 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:46.813+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:47.037 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:47.035+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:47.849 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:47.847+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:48.028 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:48.025+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:48.857 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:48.855+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:49.057 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:49.055+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:49.850 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:49.849+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:50.013 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:50.011+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:50.823 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:50.821+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:51.010 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:51.008+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:51.780 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:51.778+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:52.035 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:52.033+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:52.811 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:52.809+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:53.068 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:53.066+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:53.827 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:53.826+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:54.020 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:54.019+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:54.847 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:54.845+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:55.067 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:55.065+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:55.886 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:55.885+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:56.029 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:56.028+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:56.849 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:56.848+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:56.988 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:56.987+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:57.850 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:57.849+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:57.965 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:57.964+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:58.847 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:58.846+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:58.927 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:58.926+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:59.828 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:14:59.827+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:14:59.920 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:14:59.919+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:00.818 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:00.817+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:00.872 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:00.871+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:01.770 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:01.769+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:01.849 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:01.848+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:02.803 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:02.801+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:02.892 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:02.891+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:03.764 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:03.763+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:03.935 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:03.934+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:04.806 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:04.805+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:04.973 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:04.972+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:05.820 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:05.819+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:05.947 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:05.946+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:06.773 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:06.772+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:06.929 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:06.928+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:07.802 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:07.801+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:07.900 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:07.899+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:08.771 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:08.770+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:08.935 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:08.934+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:09.791 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:09.790+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:09.902 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:09.901+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:10.759 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:10.758+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:10.904 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:10.903+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:11.742 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:11.741+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:11.930 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:11.930+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:12.738 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:12.737+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:12.954 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:12.953+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:13.742 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:13.741+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:13.944 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:13.943+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:14.761 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:14.760+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:14.944 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:14.944+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:15.808 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:15.806+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:15.966 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:15.964+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:16.815 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:16.813+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:16.964 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:16.962+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:17.800 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:17.798+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:17.938 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:17.936+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:18.815 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:18.813+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:18.918 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:18.916+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:19.859 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:19.857+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:19.915 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:19.913+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:20.877 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:20.836+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:20.937 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:20.933+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:21.846 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:21.844+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:21.962 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:21.960+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:22.863 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:22.861+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:23.007 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:23.005+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:23.894 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:23.892+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:24.053 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:24.051+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:24.898 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:24.896+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:25.098 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:25.096+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:25.891 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:25.889+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:26.103 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:26.101+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:26.900 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:26.898+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:27.128 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:27.126+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:27.912 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:27.910+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:28.160 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:28.158+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:28.937 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:28.935+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:29.116 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:29.114+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:29.920 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:29.918+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:30.111 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:30.109+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:30.903 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:30.901+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:31.127 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:31.125+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:31.929 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:31.928+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:32.086 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:32.084+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:32.948 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:32.946+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:33.088 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:33.086+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:33.985 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:33.983+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:34.040 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:34.038+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:35.000 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:34.998+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:35.026 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:35.024+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:35.974 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:35.972+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:35.993 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:35.992+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:36.982 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:36.980+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:37.000 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:36.998+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:37.936 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:37.934+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:37.961 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:37.960+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:38.929 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:38.927+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:38.967 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:38.965+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:39.935 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:39.933+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:40.008 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:40.006+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:40.917 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:40.915+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:40.990 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:40.988+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:41.936 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:41.934+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:41.997 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:41.994+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:42.973 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:42.971+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:43.046 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:43.044+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:43.927 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:43.925+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:44.096 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:44.094+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:44.906 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:44.905+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:45.144 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:45.142+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:45.908 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:45.906+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:46.173 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:46.171+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:47.006 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:46.939+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:47.165 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:47.163+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:47.984 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:47.982+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:48.174 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:48.172+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:49.000 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:48.998+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:49.199 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:49.197+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:50.017 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:50.015+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:50.205 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:50.204+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:51.036 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:51.034+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:51.212 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:51.210+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:52.049 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:52.047+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:52.202 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:52.200+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:53.044 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:53.042+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:53.210 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:53.209+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:54.067 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:54.066+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:54.253 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:54.252+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:55.092 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:55.091+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:55.205 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:55.204+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:56.141 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:56.140+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:56.213 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:56.212+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:57.108 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:57.107+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:57.196 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:57.195+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:58.103 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:58.102+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:58.192 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:58.191+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:59.139 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:15:59.138+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:15:59.192 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:15:59.192+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:00.104 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:00.103+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:00.221 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:00.221+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:01.060 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:01.059+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:01.206 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:01.205+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:02.013 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:02.013+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:02.169 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:02.168+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:03.007 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:03.006+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:03.125 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:03.124+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:04.008 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:04.007+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:04.113 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:04.112+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:05.013 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:05.012+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:05.163 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:05.162+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:06.027 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:06.026+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:06.159 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:06.158+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:07.055 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:07.054+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:07.123 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:07.122+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:08.081 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:08.080+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:08.104 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:08.103+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:09.075 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:09.074+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:09.117 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:09.116+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:10.075 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:10.074+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:10.104 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:10.103+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:11.040 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:11.038+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:11.134 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:11.132+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:12.042 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:12.042+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:12.159 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:12.158+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:13.025 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:13.024+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:13.153 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:13.152+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:13.988 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:13.986+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:14.186 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:14.184+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:14.947 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:14.945+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:15.200 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:15.198+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:15.921 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:15.919+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:16.216 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:16.214+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:16.946 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:16.944+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:17.227 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:17.225+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:17.903 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:17.901+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:18.276 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:18.274+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:18.891 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:18.889+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:19.323 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:19.321+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:19.857 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:19.855+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:20.341 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:20.339+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:20.883 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:20.880+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:21.333 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:21.331+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:21.894 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:21.892+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:22.362 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:22.361+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:22.913 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:22.911+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:23.408 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:23.406+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:23.890 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:23.888+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:24.413 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:24.411+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:24.850 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:24.848+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:25.456 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:25.454+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:25.894 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:25.892+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:26.459 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:26.457+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:26.845 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:26.843+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:27.418 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:27.416+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:27.823 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:27.821+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:28.375 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:28.373+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:28.855 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:28.853+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:29.400 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:29.398+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:29.903 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:29.901+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:30.356 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:30.354+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:30.941 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:30.939+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:31.404 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:31.402+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:31.977 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:31.975+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:32.368 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:32.366+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:32.952 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:32.950+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:33.406 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:33.404+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:33.950 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:33.948+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:34.405 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:34.403+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:34.985 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:34.983+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:35.404 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:35.403+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:35.967 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:35.965+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:36.410 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:36.409+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:36.951 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:36.949+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:37.426 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:37.424+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:37.911 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:37.909+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:38.386 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:38.384+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:38.867 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:38.865+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:39.384 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:39.382+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:39.840 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:39.838+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:40.343 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:40.342+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:40.796 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:40.793+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:41.329 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:41.327+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:41.799 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:41.797+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:42.286 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:42.284+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:42.783 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:42.781+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:43.271 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:43.270+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:43.799 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:43.797+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:44.249 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:44.247+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:44.789 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:44.787+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:45.236 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:45.235+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:45.808 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:45.806+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:46.279 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:46.277+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:46.786 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:46.784+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:47.288 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:47.286+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:47.820 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:47.818+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:48.330 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:48.328+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:48.783 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:48.781+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:49.308 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:49.306+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:49.799 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:49.797+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:50.349 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:50.348+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:50.846 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:50.845+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:51.315 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:51.314+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:51.844 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:51.843+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:52.280 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:52.279+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:52.812 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:52.811+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:53.263 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:53.262+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:53.813 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:53.812+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:54.272 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:54.271+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:54.810 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:54.809+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:55.250 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:55.249+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:55.796 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:55.795+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:56.203 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:56.202+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:56.790 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:56.789+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:57.206 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:57.206+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:57.804 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:57.803+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:58.197 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:58.196+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:58.820 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:58.819+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:59.228 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:16:59.227+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:16:59.814 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:16:59.813+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:00.187 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:00.185+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:00.838 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:00.837+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:01.226 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:01.226+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:01.816 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:01.815+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:02.261 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:02.260+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:02.819 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:02.818+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:03.302 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:03.301+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:03.856 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:03.855+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:04.322 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:04.321+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:04.821 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:04.820+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:05.296 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:05.296+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:05.792 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:05.791+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:06.336 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:06.335+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:06.765 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:06.764+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:07.324 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:07.323+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:07.773 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:07.772+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:08.334 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:08.333+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:08.746 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:08.745+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:09.324 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:09.323+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:09.765 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:09.764+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:10.315 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:10.314+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:10.798 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:10.796+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:11.331 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:11.329+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:11.768 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:11.766+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:12.333 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:12.331+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:12.788 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:12.786+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:13.340 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:13.338+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:13.758 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:13.756+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:14.302 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:14.300+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:14.788 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:14.786+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:15.274 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:15.272+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:15.784 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:15.782+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:16.273 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:16.271+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:16.756 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:16.754+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:17.248 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:17.246+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:17.755 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:17.753+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:18.250 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:18.247+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:18.795 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:18.793+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:19.289 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:19.287+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:19.773 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:19.771+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:20.260 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:20.258+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:20.756 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:20.754+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:21.237 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:21.235+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:21.805 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:21.803+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:22.248 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:22.246+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:22.777 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:22.775+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:23.208 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:23.206+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:23.742 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:23.740+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:24.222 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:24.220+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:24.732 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:24.730+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:25.206 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:25.204+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:25.764 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:25.762+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:26.250 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:26.248+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:26.786 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:26.784+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:27.207 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:27.205+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:27.820 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:27.818+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:28.168 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:28.166+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:28.782 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:28.780+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:29.168 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:29.165+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:29.748 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:29.746+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:30.159 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:30.157+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:30.716 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:30.714+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:31.191 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:31.189+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:31.727 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:31.725+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:32.151 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:32.149+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:32.753 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:32.751+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:33.200 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:33.198+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:33.749 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:33.747+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:34.202 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:34.200+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:34.706 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:34.705+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:35.229 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:35.227+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:35.695 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:35.693+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:36.237 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:36.235+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:36.663 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:36.661+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:37.249 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:37.247+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:37.662 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:37.660+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:38.283 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:38.282+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:38.648 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:38.646+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:39.244 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:39.242+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:39.604 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:39.602+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:40.292 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:40.290+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:40.585 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:40.583+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:41.258 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:41.256+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:41.538 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:41.536+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:42.291 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:42.289+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:42.530 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:42.529+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:43.295 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:43.293+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:43.506 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:43.504+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:44.295 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:44.293+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:44.548 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:44.546+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:45.297 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:45.293+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:45.557 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:45.555+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:46.318 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:46.316+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:46.571 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:46.569+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:47.348 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:47.346+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:47.589 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:47.588+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:48.316 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:48.315+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:48.584 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:48.583+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:49.311 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:49.310+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:49.609 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:49.606+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:50.351 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:50.350+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:50.611 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:50.610+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:51.374 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:51.373+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:51.646 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:51.645+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:52.370 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:52.369+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:52.600 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:52.599+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:53.411 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:53.410+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:53.628 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:53.627+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:54.431 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:54.430+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:54.614 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:54.613+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:55.403 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:55.402+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:55.629 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:55.628+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:56.396 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:56.395+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:56.661 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:56.660+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:57.427 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:57.426+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:57.662 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:57.662+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:58.425 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:58.423+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:58.709 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:58.708+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:59.467 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:17:59.466+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:17:59.701 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:17:59.700+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:00.434 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:00.433+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:00.709 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:00.708+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:01.415 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:01.414+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:01.691 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:01.690+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:02.459 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:02.458+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:02.711 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:02.710+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:03.485 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:03.484+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:03.761 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:03.759+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:04.517 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:04.516+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:04.720 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:04.719+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:05.513 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:05.512+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:05.719 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:05.718+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:06.477 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:06.476+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:06.678 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:06.678+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:07.523 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:07.522+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:07.671 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:07.670+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:08.484 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:08.483+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:08.679 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:08.678+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:09.504 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:09.501+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:09.723 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:09.721+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:10.469 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:10.467+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:10.678 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:10.676+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:11.485 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:11.483+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:11.659 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:11.657+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:12.521 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:12.519+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:12.615 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:12.613+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:13.565 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:13.563+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:13.599 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:13.597+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:14.541 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:14.539+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:14.597 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:14.595+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:15.543 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:15.541+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:15.634 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:15.632+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:16.579 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:16.577+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:16.585 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:16.583+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:17.545 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:17.543+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:17.593 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:17.591+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:18.543 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:18.541+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:18.633 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:18.631+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:19.576 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:19.574+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:19.605 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:19.603+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:20.559 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:20.557+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:20.636 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:20.634+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:21.585 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:21.583+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:21.615 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:21.613+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:22.575 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:22.573+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:22.646 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:22.644+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:23.534 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:23.532+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:23.601 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:23.599+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:24.496 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:24.494+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:24.588 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:24.586+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:25.484 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:25.482+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:25.629 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:25.627+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:26.535 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:26.533+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:26.633 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:26.631+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:27.570 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:27.569+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:27.663 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:27.661+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:28.569 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:28.567+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:28.695 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:28.693+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:29.565 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:29.563+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:29.725 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:29.723+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:30.613 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:30.611+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:30.756 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:30.755+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:31.626 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:31.624+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:31.735 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:31.733+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:32.581 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:32.579+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:32.724 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:32.723+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:33.585 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:33.583+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:33.695 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:33.693+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:34.563 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:34.561+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:34.705 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:34.703+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:35.584 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:35.582+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:35.668 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:35.666+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:36.574 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:36.572+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:36.674 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:36.672+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:37.575 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:37.573+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:37.703 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:37.701+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:38.565 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:38.561+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:38.706 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:38.704+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:39.605 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:39.603+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:39.682 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:39.680+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:40.633 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:40.631+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:40.645 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:40.643+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:41.668 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:41.666+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:41.672 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:41.670+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:42.635 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:42.633+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:42.711 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:42.709+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:43.623 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:43.621+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:43.733 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:43.731+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:44.634 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:44.632+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:44.689 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:44.687+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:45.650 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:45.648+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:45.720 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:45.719+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:46.618 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:46.616+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:46.755 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:46.753+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:47.624 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:47.623+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:47.774 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:47.773+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:48.582 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:48.581+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:48.738 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:48.737+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:49.546 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:49.545+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:49.734 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:49.733+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:50.566 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:50.565+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:50.733 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:50.732+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:51.606 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:51.605+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:51.723 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:51.722+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:52.568 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:52.567+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:52.756 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:52.755+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:53.592 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:53.591+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:53.785 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:53.783+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:54.628 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:54.627+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:54.760 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:54.759+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:55.586 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:55.585+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:55.771 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:55.770+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:56.576 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:56.575+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:56.799 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:56.798+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:57.614 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:57.613+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:57.758 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:57.757+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:58.628 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:58.627+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:58.772 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:58.771+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:59.638 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:18:59.637+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:18:59.807 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:18:59.806+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:00.593 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:00.592+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:00.808 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:00.807+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:01.616 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:01.615+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:01.816 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:01.815+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:02.664 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:02.663+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:02.838 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:02.837+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:03.676 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:03.675+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:03.887 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:03.886+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:04.663 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:04.662+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:04.842 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:04.841+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:05.701 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:05.700+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:05.819 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:05.818+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:06.694 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:06.693+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:06.863 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:06.862+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:07.676 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:07.674+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:07.848 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:07.846+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:08.688 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:08.686+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:08.883 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:08.881+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:09.663 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:09.661+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:09.924 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:09.922+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:10.637 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:10.635+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:10.930 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:10.928+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:11.673 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:11.671+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:11.942 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:11.940+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:12.702 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:12.700+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:12.896 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:12.894+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:13.722 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:13.720+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:13.933 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:13.931+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:14.717 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:14.715+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:14.949 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:14.947+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:15.713 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:15.711+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:15.963 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:15.962+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:16.744 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:16.742+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:16.938 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:16.936+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:17.766 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:17.764+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:17.942 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:17.940+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:18.792 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:18.790+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:18.989 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:18.987+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:19.821 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:19.819+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:19.957 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:19.955+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:20.819 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:20.817+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:20.944 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:20.942+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:21.856 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:21.853+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:21.902 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:21.901+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:22.869 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:22.867+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:22.886 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:22.884+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:23.883 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:23.881+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:23.928 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:23.926+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:24.905 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:24.903+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:24.933 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:24.931+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:25.892 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:25.889+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:25.932 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:25.930+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:26.891 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:26.889+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:26.921 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:26.919+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:27.860 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:27.858+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:27.937 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:27.935+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:28.868 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:28.865+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:28.913 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:28.911+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:29.909 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:29.907+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:29.964 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:29.962+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:30.913 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:30.911+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:31.006 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:31.004+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:31.893 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:31.891+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:31.967 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:31.965+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:32.857 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:32.855+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:32.998 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:32.997+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:33.813 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:33.811+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:33.969 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:33.967+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:34.793 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:34.792+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:34.933 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:34.931+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:35.790 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:35.788+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:35.945 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:35.943+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:36.810 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:36.809+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:36.949 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:36.947+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:37.810 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:37.808+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:37.970 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:37.968+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:38.837 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:38.835+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:39.007 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:39.005+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:39.878 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:39.876+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:40.017 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:40.015+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:40.844 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:40.842+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:41.024 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:41.022+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:41.891 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:41.889+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:42.045 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:42.043+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:42.882 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:42.880+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:43.065 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:43.063+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:43.884 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:43.882+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:44.027 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:44.025+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:44.903 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:44.902+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:45.043 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:45.042+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:45.921 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:45.920+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:46.040 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:46.040+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:46.939 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:46.938+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:47.089 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:47.088+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:47.932 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:47.931+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:48.102 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:48.101+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:48.968 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:48.967+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:49.152 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:49.151+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:50.009 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:50.008+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:50.146 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:50.145+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:50.980 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:50.979+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:51.148 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:51.147+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:51.944 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:51.943+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:52.123 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:52.122+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:52.907 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:52.906+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:53.090 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:53.089+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:53.908 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:53.908+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:54.064 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:54.063+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:54.945 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:54.944+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:55.093 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:55.092+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:55.897 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:55.896+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:56.115 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:56.114+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:56.872 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:56.871+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:57.081 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:57.080+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:57.911 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:57.910+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:58.058 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:58.057+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:58.869 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:58.868+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:59.047 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:59.046+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:59.911 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:19:59.910+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:19:59.998 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:19:59.997+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:00.911 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:00.910+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:00.981 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:00.980+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:01.946 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:01.945+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:02.001 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:01.999+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:02.939 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:02.938+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:03.034 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:03.033+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:03.893 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:03.892+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:04.026 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:04.025+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:04.853 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:04.852+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:05.027 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:05.026+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:05.836 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:05.834+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:06.017 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:06.016+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:06.829 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:06.827+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:07.045 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:07.043+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:07.788 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:07.786+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:08.019 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:08.017+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:08.802 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:08.800+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:09.022 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:09.020+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:09.782 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:09.780+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:10.034 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:10.032+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:10.739 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:10.737+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:11.071 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:11.069+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:11.760 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:11.758+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:12.119 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:12.117+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:12.793 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:12.791+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:13.122 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:13.120+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:13.799 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:13.797+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:14.163 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:14.161+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:14.801 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:14.799+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:15.137 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:15.135+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:15.789 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:15.787+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:16.111 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:16.109+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:16.799 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:16.797+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:17.091 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:17.089+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:17.766 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:17.761+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:18.081 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:18.079+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:18.752 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:18.750+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:19.113 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:19.111+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:19.801 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:19.799+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:20.147 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:20.145+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:20.753 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:20.751+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:21.121 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:21.119+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:21.730 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:21.728+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:22.088 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:22.086+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:22.725 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:22.723+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:23.103 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:23.101+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:23.733 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:23.731+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:24.086 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:24.085+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:24.740 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:24.738+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:25.084 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:25.082+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:25.719 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:25.717+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:26.077 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:26.075+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:26.724 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:26.722+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:27.094 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:27.093+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:27.713 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:27.711+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:28.068 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:28.067+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:28.723 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:28.721+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:29.034 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:29.032+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:29.675 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:29.673+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:30.002 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:30.000+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:30.630 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:30.628+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:31.022 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:31.020+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:31.626 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:31.624+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:32.053 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:32.051+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:32.649 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:32.647+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:33.011 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:33.009+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:33.635 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:33.633+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:33.967 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:33.965+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:34.684 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:34.682+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:34.949 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:34.947+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:35.642 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:35.640+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:35.902 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:35.900+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:36.669 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:36.667+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:36.933 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:36.930+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:37.674 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:37.672+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:37.960 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:37.958+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:38.637 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:38.636+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:38.964 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:38.962+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:39.667 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:39.665+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:39.950 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:39.948+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:40.717 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:40.715+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:40.998 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:40.996+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:41.711 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:41.709+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:42.015 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:42.013+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:42.710 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:42.708+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:42.991 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:42.989+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:43.749 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:43.747+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:43.953 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:43.952+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:44.738 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:44.737+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:44.912 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:44.911+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:45.706 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:45.705+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:45.954 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:45.953+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:46.727 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:46.726+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:46.970 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:46.969+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:47.683 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:47.683+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:47.961 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:47.961+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:48.679 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:48.678+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:49.006 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:49.005+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:49.700 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:49.699+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:50.014 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:50.014+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:50.716 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:50.715+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:51.002 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:51.001+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:51.718 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:51.717+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:52.035 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:52.034+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:52.761 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:52.760+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:52.996 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:52.995+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:53.758 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:53.757+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:53.994 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:53.993+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:54.756 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:54.755+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:54.954 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:54.953+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:55.781 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:55.780+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:55.944 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:55.943+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:56.768 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:56.767+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:56.968 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:56.967+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:57.753 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:57.752+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:58.017 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:58.016+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:58.776 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:58.775+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:58.970 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:58.969+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:59.797 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:20:59.796+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:20:59.956 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:20:59.955+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:00.775 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:00.774+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:00.911 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:00.910+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:01.779 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:01.778+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:01.898 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:01.897+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:02.758 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:02.757+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:02.927 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:02.926+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:03.768 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:03.767+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:03.924 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:03.923+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:04.813 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:04.811+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:04.876 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:04.874+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:05.816 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:05.814+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:05.858 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:05.856+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:06.815 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:06.813+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:06.815 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:06.814+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:07.791 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:07.789+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:07.858 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:07.856+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:08.807 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:08.805+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:08.858 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:08.856+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:09.770 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:09.768+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:09.851 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:09.849+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:10.793 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:10.791+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:10.804 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:10.802+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:11.832 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:11.830+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:11.842 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:11.841+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:12.808 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:12.805+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:12.854 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:12.852+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:13.812 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:13.810+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:13.883 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:13.881+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:14.804 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:14.802+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:14.932 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:14.930+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:15.812 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:15.810+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:15.903 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:15.901+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:16.785 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:16.783+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:16.946 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:16.944+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:17.819 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:17.817+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:17.934 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:17.932+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:18.773 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:18.771+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:18.906 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:18.904+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:19.818 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:19.816+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:19.912 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:19.910+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:20.786 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:20.784+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:20.915 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:20.914+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:21.814 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:21.812+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:21.908 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:21.906+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:22.818 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:22.816+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:22.912 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:22.910+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:23.773 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:23.771+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:23.960 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:23.958+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:24.750 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:24.748+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:24.919 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:24.917+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:25.743 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:25.742+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:25.872 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:25.870+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:26.713 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:26.710+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:26.883 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:26.881+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:27.684 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:27.682+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:27.911 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:27.909+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:28.657 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:28.655+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:28.914 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:28.912+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:29.691 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:29.689+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:29.955 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:29.953+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:30.710 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:30.708+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:30.958 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:30.956+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:31.690 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:31.687+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:31.950 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:31.948+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:32.640 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:32.638+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:32.917 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:32.916+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:33.604 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:33.602+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:33.875 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:33.873+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:34.651 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:34.649+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:34.905 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:34.903+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:35.629 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:35.627+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:35.918 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:35.916+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:36.613 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:36.611+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:36.947 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:36.946+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:37.594 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:37.592+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:37.912 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:37.910+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:38.577 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:38.575+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:38.906 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:38.904+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:39.569 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:39.567+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:39.955 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:39.952+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:40.602 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:40.600+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:40.953 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:40.951+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:41.618 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:41.616+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:41.985 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:41.984+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:42.668 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:42.667+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:43.003 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:43.002+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:43.640 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:43.639+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:44.012 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:44.010+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:44.593 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:44.592+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:45.012 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:45.011+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:45.634 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:45.630+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:45.985 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:45.984+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:46.670 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:46.669+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:46.993 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:46.992+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:47.679 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:47.678+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:48.038 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:48.037+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:48.661 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:48.660+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:49.001 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:49.000+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:49.630 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:49.629+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:50.044 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:50.043+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:50.657 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:50.656+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:51.016 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:51.015+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:51.626 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:51.626+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:51.987 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:51.984+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:52.659 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:52.658+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:52.949 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:52.948+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:53.702 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:53.701+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:53.918 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:53.917+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:54.688 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:54.687+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:54.930 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:54.929+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:55.737 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:55.736+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:55.917 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:55.916+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:56.719 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:56.718+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:56.943 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:56.942+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:57.707 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:57.706+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:57.984 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:57.982+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:58.707 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:58.706+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:59.022 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:21:59.022+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:21:59.663 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:21:59.662+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:00.002 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:00.000+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:00.699 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:00.697+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:00.980 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:00.979+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:01.701 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:01.699+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:01.955 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:01.954+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:02.690 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:02.689+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:02.942 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:02.940+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:03.717 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:03.714+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:03.936 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:03.935+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:04.696 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:04.695+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:04.916 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:04.915+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:05.743 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:05.741+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:05.956 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:05.953+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:06.728 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:06.726+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:06.952 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:06.950+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:07.710 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:07.708+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:07.931 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:07.929+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:08.741 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:08.739+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:08.948 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:08.946+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:09.779 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:09.777+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:09.927 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:09.924+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:10.804 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:10.802+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:10.880 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:10.879+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:11.789 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:11.787+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:11.897 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:11.895+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:12.785 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:12.782+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:12.908 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:12.906+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:13.738 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:13.736+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:13.953 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:13.951+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:14.769 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:14.767+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:14.954 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:14.952+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:15.746 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:15.745+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:15.935 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:15.933+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:16.764 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:16.762+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:16.889 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:16.887+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:17.776 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:17.774+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:17.884 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:17.882+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:18.743 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:18.741+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:18.934 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:18.932+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:19.790 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:19.788+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:19.942 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:19.941+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:20.820 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:20.818+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:20.922 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:20.921+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:21.830 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:21.828+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:21.956 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:21.954+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:22.810 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:22.808+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:22.998 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:22.996+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:23.764 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:23.762+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:23.965 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:23.963+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:24.814 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:24.812+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:24.929 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:24.927+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:25.813 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:25.811+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:25.893 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:25.891+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:26.802 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:26.800+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:26.932 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:26.931+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:27.788 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:27.786+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:27.978 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:27.976+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:28.818 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:28.816+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:28.992 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:28.990+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:29.827 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:29.825+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:29.980 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:29.977+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:30.855 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:30.853+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:30.947 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:30.945+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:31.880 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:31.878+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:31.949 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:31.947+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:32.904 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:32.902+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:32.951 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:32.949+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:33.920 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:33.918+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:33.933 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:33.931+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:34.883 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:34.881+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:34.964 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:34.962+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:35.885 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:35.882+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:35.981 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:35.979+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:36.902 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:36.899+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:36.982 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:36.981+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:37.922 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:37.919+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:37.986 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:37.984+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:38.917 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:38.916+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:38.976 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:38.975+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:39.890 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:39.889+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:39.944 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:39.942+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:40.867 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:40.865+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:40.944 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:40.943+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:41.871 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:41.870+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:41.896 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:41.895+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:42.875 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:42.874+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:42.886 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:42.885+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:43.904 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:43.903+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:43.911 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:43.910+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:44.896 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:44.895+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:44.960 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:44.959+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:45.869 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:45.868+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:45.916 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:45.915+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:46.913 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:46.912+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:46.935 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:46.934+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:47.918 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:47.916+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:47.961 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:47.960+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:48.955 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:48.954+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:48.981 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:48.980+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:49.934 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:49.933+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:49.961 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:49.960+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:50.917 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:50.916+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:50.940 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:50.939+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:51.901 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:51.900+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:51.944 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:51.943+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:52.895 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:52.894+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:52.916 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:52.915+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:53.887 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:53.886+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:53.922 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:53.921+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:54.887 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:54.886+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:54.901 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:54.900+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:55.869 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:55.868+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:55.893 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:55.892+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:56.853 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:56.852+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:56.937 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:56.936+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:57.828 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:57.827+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:57.929 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:57.928+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:58.867 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:58.866+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:58.889 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:58.889+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:59.883 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:22:59.882+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:22:59.919 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:22:59.918+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:00.907 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:00.906+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:01.006 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:00.932+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:01.929 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:01.927+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:01.936 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:01.934+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:02.893 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:02.891+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:02.907 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:02.905+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:03.875 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:03.873+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:03.909 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:03.907+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:04.903 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:04.901+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:04.921 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:04.919+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:05.946 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:05.944+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:05.972 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:05.970+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:06.947 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:06.945+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:06.950 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:06.948+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:07.927 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:07.925+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:07.969 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:07.967+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:08.904 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:08.901+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:08.990 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:08.988+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:09.917 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:09.915+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:09.993 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:09.991+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:10.891 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:10.889+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:10.963 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:10.961+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:11.867 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:11.865+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:11.990 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:11.988+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:12.850 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:12.848+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:13.032 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:13.030+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:13.824 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:13.822+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:14.059 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:14.057+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:14.833 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:14.831+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:15.109 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:15.107+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:15.850 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:15.848+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:16.061 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:16.059+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:16.899 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:16.897+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:17.033 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:17.031+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:17.912 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:17.910+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:18.064 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:18.062+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:18.902 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:18.899+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:19.043 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:19.041+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:19.907 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:19.905+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:20.055 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:20.053+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:20.871 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:20.869+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:21.057 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:21.055+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:21.875 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:21.873+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:22.085 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:22.083+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:22.828 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:22.827+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:23.063 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:23.061+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:23.785 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:23.783+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:24.085 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:24.083+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:24.759 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:24.757+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:25.092 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:25.090+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:25.717 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:25.715+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:26.075 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:26.073+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:26.675 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:26.673+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:27.028 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:27.025+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:27.654 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:27.652+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:27.993 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:27.991+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:28.662 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:28.660+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:28.965 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:28.963+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:29.686 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:29.684+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:29.987 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:29.985+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:30.652 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:30.650+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:30.989 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:30.987+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:31.609 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:31.607+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:31.968 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:31.967+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:32.627 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:32.625+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:33.010 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:33.009+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:33.652 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:33.650+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:34.056 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:34.054+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:34.679 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:34.677+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:35.008 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:35.006+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:35.635 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:35.633+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:35.959 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:35.957+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:36.604 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:36.602+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:36.993 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:36.991+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:37.560 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:37.558+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:37.949 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:37.947+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:38.516 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:38.514+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:38.996 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:38.994+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:39.467 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:39.465+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:40.013 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:40.012+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:40.504 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:40.503+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:41.050 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:41.049+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:41.541 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:41.540+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:42.040 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:42.039+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:42.531 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:42.530+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:43.066 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:43.065+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:43.564 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:43.563+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:44.111 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:44.109+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:44.538 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:44.537+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:45.094 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:45.093+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:45.558 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:45.557+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:46.134 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:46.133+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:46.532 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:46.531+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:47.093 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:47.092+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:47.544 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:47.543+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:48.122 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:48.121+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:48.542 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:48.541+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:49.104 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:49.103+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:49.571 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:49.570+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:50.145 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:50.144+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:50.564 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:50.563+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:51.118 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:51.117+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:51.601 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:51.600+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:52.134 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:52.133+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:52.605 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:52.603+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:53.152 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:53.151+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:53.645 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:53.644+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:54.176 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:54.175+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:54.597 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:54.596+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:55.167 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:55.166+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:55.642 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:55.641+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:56.216 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:56.215+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:56.593 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:56.593+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:57.189 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:57.188+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:57.602 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:57.601+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:58.218 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:58.217+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:58.573 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:58.572+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:59.245 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:23:59.244+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:23:59.598 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:23:59.597+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:00.218 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:00.216+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:00.578 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:00.576+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:01.233 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:01.231+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:01.557 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:01.555+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:02.208 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:02.206+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:02.517 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:02.515+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:03.181 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:03.179+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:03.482 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:03.480+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:04.180 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:04.178+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:04.496 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:04.494+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:05.199 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:05.197+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:05.510 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:05.508+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:06.182 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:06.180+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:06.487 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:06.485+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:07.194 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:07.192+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:07.525 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:07.523+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:08.199 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:08.197+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:08.535 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:08.533+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:09.179 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:09.177+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:09.554 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:09.552+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:10.206 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:10.204+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:10.587 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:10.586+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:11.158 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:11.156+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:11.543 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:11.541+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:12.138 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:12.136+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:12.559 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:12.557+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:13.121 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:13.119+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:13.587 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:13.585+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:14.143 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:14.141+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:14.631 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:14.629+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:15.125 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:15.123+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:15.627 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:15.625+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:16.091 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:16.089+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:16.630 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:16.628+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:17.055 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:17.053+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:17.650 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:17.648+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:18.058 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:18.056+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:18.684 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:18.682+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:19.047 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:19.045+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:19.702 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:19.700+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:20.067 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:20.065+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:20.712 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:20.710+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:21.097 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:21.095+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:21.730 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:21.728+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:22.121 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:22.119+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:22.695 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:22.693+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:23.109 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:23.108+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:23.669 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:23.667+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:24.127 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:24.125+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:24.694 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:24.692+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:25.168 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:25.166+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:25.736 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:25.734+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:26.119 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:26.117+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:26.752 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:26.750+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:27.137 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:27.135+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:27.747 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:27.746+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:28.183 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:28.181+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:28.790 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:28.788+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:29.188 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:29.186+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:29.773 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:29.771+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:30.191 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:30.189+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:30.735 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:30.733+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:31.205 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:31.203+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:31.691 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:31.689+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:32.241 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:32.240+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:32.702 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:32.700+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:33.271 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:33.269+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:33.728 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:33.726+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:34.258 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:34.256+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:34.717 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:34.715+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:35.292 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:35.290+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:35.703 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:35.701+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:36.267 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:36.265+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:36.697 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:36.696+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:37.270 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:37.268+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:37.654 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:37.652+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:38.247 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:38.246+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:38.612 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:38.611+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:39.223 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:39.222+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:39.608 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:39.607+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:40.200 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:40.198+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:40.604 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:40.603+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:41.178 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:41.177+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:41.640 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:41.639+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:42.169 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:42.168+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:42.656 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:42.655+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:43.211 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:43.210+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:43.670 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:43.669+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:44.237 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:44.236+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:44.657 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:44.656+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:45.276 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:45.275+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:45.696 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:45.695+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:46.299 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:46.298+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:46.726 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:46.725+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:47.330 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:47.329+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:47.726 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:47.725+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:48.370 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:48.369+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:48.677 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:48.676+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:49.404 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:49.403+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:49.670 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:49.669+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:50.421 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:50.420+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:50.640 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:50.639+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:51.389 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:51.388+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:51.641 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:51.640+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:52.354 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:52.353+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:52.672 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:52.671+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:53.312 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:53.311+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:53.716 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:53.715+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:54.359 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:54.358+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:54.688 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:54.687+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:55.323 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:55.322+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:55.642 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:55.641+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:56.367 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:56.366+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:56.617 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:56.616+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:57.330 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:57.329+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:57.600 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:57.599+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:58.311 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:58.310+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:58.641 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:58.639+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:59.312 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:24:59.310+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:24:59.692 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:24:59.690+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:00.353 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:00.351+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:00.690 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:00.688+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:01.334 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:01.332+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:01.713 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:01.712+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:02.337 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:02.335+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:02.695 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:02.693+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:03.304 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:03.302+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:03.674 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:03.672+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:04.304 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:04.302+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:04.652 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:04.650+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:05.257 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:05.255+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:05.616 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:05.614+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:06.254 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:06.252+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:06.661 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:06.659+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:07.253 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:07.251+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:07.654 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:07.652+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:08.296 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:08.294+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:08.659 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:08.658+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:09.310 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:09.308+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:09.698 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:09.696+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:10.297 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:10.295+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:10.696 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:10.694+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:11.260 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:11.258+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:11.702 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:11.700+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:12.229 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:12.227+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:12.653 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:12.651+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:13.184 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:13.182+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:13.629 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:13.627+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:14.220 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:14.217+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:14.596 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:14.594+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:15.183 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:15.180+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:15.612 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:15.610+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:16.156 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:16.154+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:16.573 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:16.571+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:17.148 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:17.146+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:17.570 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:17.568+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:18.163 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:18.161+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:18.558 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:18.556+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:19.156 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:19.154+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:19.559 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:19.558+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:20.173 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:20.171+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:20.530 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:20.528+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:21.138 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:21.136+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:21.542 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:21.540+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:22.151 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:22.149+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:22.591 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:22.589+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:23.191 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:23.189+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:23.559 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:23.556+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:24.224 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:24.222+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:24.550 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:24.548+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:25.274 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:25.272+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:25.594 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:25.592+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:26.254 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:26.252+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:26.615 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:26.614+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:27.219 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:27.218+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:27.576 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:27.574+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:28.213 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:28.212+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:28.585 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:28.583+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:29.258 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:29.256+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:29.555 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:29.553+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:30.224 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:30.222+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:30.596 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:30.594+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:31.182 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:31.180+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:31.620 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:31.618+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:32.159 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:32.157+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:32.626 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:32.624+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:33.183 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:33.181+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:33.589 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:33.587+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:34.205 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:34.203+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:34.584 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:34.581+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:35.211 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:35.209+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:35.604 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:35.602+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:36.237 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:36.235+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:36.608 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:36.606+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:37.189 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:37.187+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:37.609 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:37.608+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:38.200 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:38.199+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:38.642 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:38.641+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:39.170 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:39.170+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:39.635 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:39.634+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:40.131 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:40.130+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:40.617 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:40.616+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:41.087 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:41.086+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:41.637 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:41.636+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:42.045 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:42.044+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:42.631 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:42.630+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:43.005 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:43.004+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:43.661 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:43.660+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:44.019 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:44.018+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:44.614 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:44.613+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:45.038 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:45.037+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:45.628 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:45.627+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:46.021 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:46.020+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:46.660 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:46.659+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:47.045 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:47.044+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:47.703 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:47.702+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:48.028 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:48.027+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:48.732 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:48.731+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:49.000 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:48.999+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:49.773 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:49.772+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:50.008 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:50.007+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:50.734 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:50.733+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:51.023 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:51.022+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:51.737 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:51.736+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:52.012 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:52.011+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:52.705 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:52.705+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:53.039 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:53.038+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:53.740 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:53.739+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:54.086 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:54.085+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:54.770 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:54.769+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:55.127 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:55.126+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:55.786 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:55.785+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:56.078 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:56.077+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:56.773 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:56.772+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:57.093 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:57.091+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:57.790 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:57.788+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:58.053 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:58.051+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:58.807 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:58.805+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:59.015 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:25:59.013+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:25:59.787 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:25:59.785+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:00.053 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:00.050+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:00.802 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:00.800+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:01.029 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:01.027+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:01.774 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:01.771+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:02.018 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:02.016+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:02.795 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:02.793+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:02.979 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:02.977+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:03.762 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:03.760+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:03.956 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:03.954+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:04.714 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:04.712+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:04.973 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:04.971+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:05.666 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:05.665+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:05.984 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:05.982+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:06.640 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:06.638+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:06.984 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:06.982+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:07.593 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:07.591+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:08.031 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:08.029+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:08.635 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:08.634+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:09.033 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:09.031+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:09.638 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:09.636+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:09.987 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:09.985+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:10.656 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:10.654+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:10.944 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:10.942+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:11.700 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:11.698+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:11.921 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:11.919+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:12.692 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:12.690+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:12.932 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:12.930+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:13.653 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:13.651+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:13.940 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:13.938+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:14.633 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:14.630+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:14.904 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:14.902+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:15.677 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:15.675+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:15.882 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:15.880+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:16.638 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:16.636+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:16.846 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:16.844+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:17.622 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:17.620+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:17.860 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:17.858+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:18.639 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:18.637+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:18.842 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:18.840+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:19.663 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:19.661+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:19.803 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:19.801+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:20.637 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:20.635+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:20.783 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:20.781+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:21.597 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:21.595+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:21.814 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:21.812+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:22.589 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:22.587+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:22.813 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:22.811+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:23.601 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:23.599+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:23.780 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:23.778+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:24.568 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:24.566+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:24.765 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:24.764+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:25.597 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:25.595+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:25.792 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:25.790+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:26.615 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:26.613+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:26.778 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:26.776+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:27.578 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:27.576+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:27.823 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:27.821+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:28.603 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:28.601+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:28.734 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T23:26:28.732+0000 7f994396e700 0 -- 172.21.15.167:0/1156974075 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f992803a600 msgr2=0x7f9928046110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-05T23:26:28.832 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:28.830+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:29.637 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:29.635+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:29.815 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:29.813+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:30.590 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:30.588+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:30.835 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:30.833+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:31.593 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:31.591+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:31.834 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:31.832+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:32.553 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:32.551+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:32.806 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:32.804+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:33.557 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:33.555+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:33.847 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:33.845+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:34.510 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:34.507+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:34.843 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:34.841+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:35.517 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:35.516+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:35.887 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:35.886+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:36.472 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:36.471+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:36.917 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:36.916+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:37.514 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:37.513+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:37.918 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:37.917+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:38.533 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:38.532+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:38.914 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:38.913+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:39.552 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:39.551+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:39.897 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:39.896+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:40.515 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:40.514+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:40.913 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:40.912+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:41.468 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:41.467+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:41.889 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:41.888+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:42.457 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:42.456+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:42.850 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:42.849+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:43.478 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:43.477+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:43.896 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:43.895+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:44.469 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:44.468+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:44.936 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:44.936+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:45.492 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:45.491+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:45.986 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:45.986+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:46.498 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:46.497+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:47.036 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:47.036+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:47.525 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:47.524+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:48.018 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:48.017+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:48.518 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:48.517+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:49.029 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:49.028+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:49.519 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:49.518+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:50.006 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:50.005+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:50.547 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:50.546+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:51.054 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:51.053+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:51.512 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:51.511+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:52.078 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:52.030+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:52.539 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:52.538+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:53.079 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:53.078+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:53.557 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:53.554+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:54.098 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:54.097+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:54.561 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:54.560+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:55.090 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:55.089+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:55.560 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:55.558+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:56.051 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:56.049+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:56.577 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:56.575+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:57.005 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:57.003+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:57.540 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:57.538+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:58.032 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:58.031+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:58.573 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:58.571+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:58.997 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:58.995+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:59.557 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:26:59.555+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:26:59.987 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:26:59.985+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:00.547 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:00.545+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:00.949 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:00.948+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:01.530 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:01.528+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:01.975 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:01.973+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:02.579 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:02.577+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:03.001 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:02.997+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:03.621 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:03.617+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:04.023 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:04.021+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:04.604 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:04.602+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:05.021 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:05.019+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:05.617 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:05.615+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:06.029 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:06.027+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:06.588 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:06.586+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:07.073 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:07.071+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:07.628 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:07.627+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:08.054 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:08.052+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:08.646 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:08.644+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:09.052 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:09.050+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:09.657 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:09.655+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:10.093 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:10.091+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:11.754 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:10.631+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:11.754 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:11.086+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:11.755 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:11.660+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:12.105 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:12.103+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:12.654 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:12.651+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:13.081 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:13.079+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:13.646 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:13.644+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:14.047 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:14.045+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:14.605 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:14.603+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:15.073 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:15.071+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:15.578 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:15.577+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:16.032 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:16.030+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:16.612 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:16.610+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:17.059 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:17.057+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:17.605 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:17.603+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:18.022 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:18.020+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:18.656 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:18.654+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:19.021 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:19.019+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:19.634 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:19.632+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:20.063 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:20.061+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:20.677 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:20.675+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:21.068 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:21.066+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:21.726 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:21.724+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:22.069 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:22.067+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:22.770 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:22.768+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:23.023 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:23.021+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:23.762 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:23.760+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:24.034 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:24.032+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:24.757 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:24.755+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:25.079 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:25.077+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:25.787 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:25.786+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:26.101 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:26.100+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:26.821 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:26.819+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:27.077 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:27.075+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:27.798 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:27.796+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:28.124 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:28.122+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:28.773 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:28.771+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:29.097 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:29.095+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:29.790 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:29.788+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:30.103 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:30.102+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:30.748 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:30.746+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:31.116 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:31.114+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:31.740 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:31.739+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:32.098 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:32.097+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:32.788 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:32.786+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:33.068 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:33.066+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:33.769 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:33.768+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:34.094 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:34.093+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:34.748 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:34.747+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:35.108 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:35.108+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:35.780 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:35.779+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:36.091 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:36.091+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:36.779 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:36.779+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:37.054 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:37.053+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:37.766 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:37.765+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:38.071 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:38.070+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:38.776 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:38.775+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:39.105 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:39.104+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:39.760 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:39.759+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:40.120 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:40.119+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:40.778 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:40.777+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:41.135 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:41.134+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:41.780 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:41.779+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:42.086 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:42.085+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:42.784 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:42.783+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:43.121 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:43.120+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:43.768 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:43.767+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:44.119 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:44.118+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:44.724 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:44.723+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:45.158 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:45.157+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:45.703 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:45.702+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:46.196 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:46.195+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:46.724 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:46.723+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:47.160 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:47.159+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:47.680 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:47.680+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:48.194 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:48.193+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:48.656 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:48.655+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:49.243 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:49.242+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:49.647 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:49.646+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:50.283 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:50.282+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:50.677 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:50.676+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:51.310 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:51.309+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:51.713 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:51.712+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:52.361 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:52.360+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:52.708 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:52.707+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:53.340 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:53.339+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:53.680 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:53.678+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:54.324 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:54.322+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:54.645 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:54.643+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:55.301 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:55.300+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:55.684 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:55.682+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:56.345 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:56.343+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:56.709 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:56.707+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:57.383 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:57.381+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:57.750 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:57.748+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:58.339 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:58.337+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:58.768 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:58.766+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:59.303 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:27:59.301+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:27:59.746 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:27:59.744+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:00.327 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:00.325+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:00.716 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:00.714+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:01.307 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:01.305+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:01.758 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:01.756+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:02.320 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:02.318+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:02.786 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:02.784+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:03.349 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:03.347+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:03.818 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:03.816+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:04.376 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:04.374+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:04.834 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:04.832+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:05.362 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:05.360+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:05.835 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:05.833+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:06.321 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:06.319+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:06.842 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:06.840+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:07.336 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:07.334+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:07.856 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:07.854+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:08.311 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:08.309+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:08.838 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:08.836+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:09.332 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:09.330+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:09.862 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:09.860+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:10.340 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:10.338+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:10.893 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:10.891+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:11.377 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:11.375+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:11.935 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:11.933+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:12.387 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:12.385+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:12.918 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:12.916+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:13.435 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:13.433+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:13.910 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:13.908+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:14.434 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:14.432+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:14.882 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:14.880+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:15.407 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:15.405+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:15.927 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:15.925+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:16.426 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:16.425+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:16.952 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:16.950+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:17.476 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:17.474+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:17.980 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:17.977+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:18.481 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:18.479+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:18.964 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:18.962+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:19.473 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:19.471+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:19.920 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:19.918+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:20.485 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:20.483+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:20.970 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:20.968+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:21.522 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:21.520+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:21.988 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:21.986+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:22.569 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:22.567+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:23.032 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:23.030+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:23.538 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:23.536+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:24.015 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:24.013+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:24.548 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:24.547+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:24.975 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:24.973+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:25.548 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:25.546+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:25.962 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:25.960+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:26.568 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:26.566+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:26.960 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:26.958+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:27.591 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:27.589+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:27.926 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:27.925+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:28.543 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:28.541+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:28.939 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:28.937+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:29.585 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:29.583+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:29.893 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:29.892+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:30.555 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:30.553+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:30.846 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:30.844+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:31.511 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:31.510+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:31.895 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:31.894+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:32.496 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:32.495+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:32.925 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:32.923+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:33.513 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:33.512+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:33.957 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:33.956+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:34.550 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:34.549+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:34.975 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:34.974+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:35.582 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:35.581+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:35.957 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:35.957+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:36.622 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:36.621+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:36.957 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:36.957+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:37.660 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:37.659+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:37.977 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:37.976+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:38.666 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:38.665+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:38.938 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:38.937+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:39.694 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:39.693+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:39.925 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:39.925+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:40.690 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:40.689+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:40.925 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:40.924+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:41.693 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:41.692+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:41.918 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:41.917+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:42.743 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:42.742+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:42.870 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:42.869+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:43.756 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:43.755+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:43.892 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:43.891+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:44.737 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:44.736+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:44.910 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:44.909+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:45.745 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:45.744+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:45.886 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:45.885+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:46.786 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:46.784+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:46.852 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:46.851+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:47.796 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:47.795+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:47.805 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:47.804+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:48.769 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:48.768+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:48.806 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:48.805+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:49.761 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:49.760+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:49.777 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:49.777+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:50.790 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:50.789+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:50.794 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:50.793+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:51.756 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:51.754+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:51.788 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:51.786+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:52.733 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:52.731+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:52.804 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:52.802+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:53.732 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:53.730+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:53.786 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:53.784+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:54.755 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:54.754+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:54.773 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:54.771+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:55.731 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:55.729+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:55.746 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:55.744+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:56.761 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:56.759+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:56.765 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:56.763+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:57.806 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:57.804+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:57.807 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:57.806+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:58.817 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:58.816+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:58.834 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:58.832+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:59.823 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:28:59.821+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:28:59.868 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:28:59.866+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:00.805 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:00.803+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:00.905 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:00.903+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:01.808 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:01.805+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:01.940 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:01.938+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:02.838 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:02.836+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:02.919 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:02.917+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:03.869 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:03.867+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:03.895 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:03.893+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:04.887 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:04.885+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:04.946 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:04.944+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:05.903 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:05.901+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:05.948 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:05.946+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:06.925 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:06.923+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:06.944 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:06.942+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:07.918 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:07.917+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:07.989 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:07.987+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:08.898 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:08.896+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:09.015 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:09.013+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:09.875 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:09.873+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:09.989 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:09.988+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:10.898 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:10.896+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:10.961 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:10.959+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:11.944 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:11.942+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:11.968 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:11.966+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:12.960 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:12.958+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:12.968 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:12.966+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:13.956 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:13.954+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:14.002 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:14.000+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:14.963 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:14.961+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:15.014 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:15.012+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:15.956 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:15.954+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:15.972 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:15.970+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:16.956 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:16.954+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:16.989 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:16.987+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:17.978 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:17.976+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:18.037 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:18.036+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:18.940 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:18.938+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:19.053 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:19.052+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:19.914 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:19.912+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:20.080 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:20.078+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:20.928 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:20.926+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:21.054 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:21.052+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:21.941 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:21.939+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:22.042 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:22.040+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:22.899 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:22.897+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:23.043 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:23.041+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:23.922 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:23.920+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:24.020 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:24.017+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:24.949 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:24.947+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:25.015 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:25.013+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:25.932 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:25.930+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:26.043 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:26.041+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:26.911 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:26.909+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:26.995 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:26.993+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:27.907 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:27.905+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:27.970 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:27.968+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:28.892 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:28.890+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:28.961 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:28.960+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:29.880 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:29.879+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:29.933 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:29.932+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:30.874 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:30.873+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:30.977 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:30.976+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:31.835 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:31.834+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:31.989 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:31.989+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:32.860 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:32.859+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:32.957 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:32.956+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:33.904 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:33.903+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:33.916 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:33.915+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:34.905 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:34.904+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:34.947 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:34.947+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:35.938 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:35.937+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:35.984 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:35.983+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:36.964 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:36.963+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:36.992 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:36.991+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:37.967 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:37.966+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:37.979 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:37.978+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:38.953 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:38.952+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:38.961 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:38.960+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:39.944 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:39.943+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:39.945 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:39.945+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:40.901 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:40.900+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:40.930 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:40.929+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:41.853 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:41.853+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:41.977 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:41.976+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:42.884 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:42.883+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:42.958 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:42.957+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:43.848 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:43.848+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:43.965 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:43.964+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:44.889 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:44.888+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:44.953 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:44.953+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:45.925 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:45.924+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:45.944 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:45.944+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:46.883 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:46.882+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:46.966 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:46.965+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:47.907 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:47.906+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:48.014 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:48.013+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:48.871 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:48.870+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:48.992 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:48.991+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:49.877 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:49.875+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:50.009 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:50.007+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:50.889 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:50.887+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:51.016 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:51.014+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:51.840 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:51.838+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:52.035 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:52.033+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:52.844 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:52.842+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:53.006 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:53.004+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:53.889 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:53.887+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:53.973 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:53.971+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:54.910 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:54.908+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:55.001 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:55.000+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:55.957 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:55.955+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:55.975 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:55.973+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:56.958 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:56.956+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:56.972 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:56.970+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:57.971 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:57.969+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:57.971 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:57.969+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:59.005 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:59.002+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:59.019 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:29:59.017+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:29:59.977 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:29:59.975+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:00.011 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:00.009+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:00.961 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:00.960+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:00.965 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:00.963+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:01.946 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:01.944+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:01.985 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:01.983+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:02.924 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:02.922+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:02.980 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:02.978+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:03.906 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:03.905+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:03.962 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:03.960+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:04.866 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:04.864+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:04.925 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:04.923+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:05.889 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:05.887+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:05.915 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:05.914+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:06.880 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:06.878+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:06.939 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:06.937+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:07.885 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:07.883+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:07.939 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:07.937+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:08.885 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:08.883+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:08.915 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:08.913+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:09.883 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:09.881+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:09.935 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:09.933+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:10.881 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:10.879+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:10.895 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:10.893+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:11.886 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:11.884+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:11.915 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:11.913+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:12.838 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:12.836+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:12.905 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:12.903+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:13.843 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:13.841+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:13.903 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:13.901+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:14.871 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:14.869+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:14.890 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:14.888+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:15.872 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:15.870+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:15.914 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:15.912+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:16.903 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:16.901+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:16.911 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:16.909+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:17.919 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:17.917+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:17.947 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:17.945+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:18.900 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:18.898+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:18.960 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:18.958+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:19.866 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:19.864+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:20.010 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:20.008+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:20.870 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:20.868+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:20.963 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:20.961+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:21.822 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:21.821+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:21.968 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:21.967+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:22.858 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:22.857+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:22.923 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:22.921+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:23.898 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:23.896+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:23.912 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:23.911+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:24.887 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:24.884+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:24.958 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:24.956+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:25.848 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:25.846+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:25.935 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:25.933+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:26.823 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:26.821+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:26.936 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:26.934+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:27.813 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:27.812+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:27.949 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:27.948+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:28.801 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:28.800+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:28.957 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:28.956+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:29.802 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:29.801+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:29.987 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:29.986+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:30.771 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:30.770+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:30.944 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:30.943+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:31.763 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:31.762+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:31.994 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:31.993+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:32.807 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:32.806+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:32.996 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:32.995+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:33.772 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:33.771+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:34.002 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:34.001+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:34.811 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:34.810+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:34.989 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:34.988+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:35.841 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:35.840+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:35.991 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:35.990+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:36.811 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:36.810+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:37.010 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:37.010+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:37.851 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:37.850+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:38.052 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:38.052+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:38.898 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:38.897+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:39.074 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:39.074+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:39.901 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:39.900+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:40.029 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:40.028+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:40.878 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:40.877+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:41.060 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:41.059+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:41.902 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:41.902+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:42.045 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:42.042+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:42.914 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:42.913+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:43.007 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:43.006+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:43.889 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:43.888+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:43.969 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:43.968+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:44.927 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:44.926+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:44.945 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:44.944+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:45.896 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:45.895+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:45.917 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:45.916+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:46.916 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:46.915+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:46.961 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:46.958+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:47.923 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:47.921+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:47.938 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:47.937+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:48.907 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:48.905+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:48.908 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:48.907+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:49.873 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:49.871+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:49.955 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:49.953+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:50.914 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:50.912+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:51.003 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:51.001+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:51.933 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:51.931+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:52.027 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:52.025+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:52.894 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:52.892+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:52.997 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:52.995+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:53.909 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:53.907+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:54.000 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:53.998+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:54.901 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:54.899+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:55.015 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:55.013+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:55.919 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:55.917+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:56.057 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:56.055+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:56.908 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:56.906+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:57.087 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:57.085+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:57.872 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:57.870+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:58.128 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:58.126+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:58.840 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:58.838+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:59.147 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:30:59.145+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:30:59.807 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:30:59.805+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:00.120 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:00.119+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:00.780 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:00.778+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:01.106 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:01.104+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:01.761 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:01.759+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:02.154 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:02.153+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:02.802 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:02.800+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:03.116 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:03.114+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:03.777 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:03.775+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:04.078 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:04.075+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:04.750 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:04.749+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:05.089 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:05.087+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:05.768 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:05.766+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:06.055 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:06.053+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:06.770 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:06.768+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:07.103 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:07.101+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:07.762 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:07.760+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:08.057 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:08.055+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:08.784 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:08.782+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:09.100 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:09.098+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:09.819 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:09.817+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:10.125 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:10.123+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:10.831 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:10.829+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:11.151 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:11.101+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:11.794 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:11.792+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:12.145 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:12.143+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:12.842 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:12.840+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:13.112 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:13.110+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:13.863 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:13.861+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:14.080 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:14.079+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:14.861 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:14.859+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:15.121 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:15.119+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:15.838 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:15.836+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:16.107 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:16.105+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:16.817 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:16.815+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:17.096 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:17.095+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:17.774 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:17.772+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:18.116 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:18.115+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:18.793 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:18.791+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:19.099 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:19.097+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:19.842 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:19.840+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:20.061 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:20.059+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:20.803 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:20.801+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:21.038 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:21.036+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:21.842 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:21.840+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:22.024 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:22.022+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:22.885 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:22.883+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:22.993 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:22.991+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:23.912 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:23.910+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:24.030 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:24.028+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:24.885 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:24.881+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:25.016 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:25.015+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:25.864 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:25.863+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:26.020 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:26.019+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:26.908 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:26.907+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:26.982 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:26.981+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:27.893 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:27.893+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:28.030 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:28.029+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:28.943 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:28.942+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:29.047 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:29.046+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:29.973 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:29.972+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:30.087 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:30.086+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:30.997 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:30.996+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:31.054 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:31.053+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:31.991 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:31.991+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:32.059 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:32.057+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:32.986 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:32.985+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:33.043 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:33.042+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:33.938 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:33.937+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:34.080 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:34.079+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:34.958 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:34.957+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:35.077 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:35.077+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:35.954 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:35.953+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:36.087 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:36.087+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:36.910 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:36.909+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:37.059 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:37.058+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:37.957 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:37.956+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:38.035 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:38.034+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:38.946 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:38.945+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:39.033 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:39.030+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:39.912 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:39.911+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:40.016 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:40.015+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:40.868 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:40.867+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:41.021 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:41.020+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:41.910 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:41.909+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:42.070 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:42.069+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:42.863 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:42.862+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:43.108 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:43.107+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:43.829 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:43.828+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:44.110 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:44.109+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:44.799 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:44.799+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:45.101 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:45.100+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:45.844 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:45.842+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:46.147 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:46.145+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:46.845 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:46.843+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:47.171 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:47.169+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:47.894 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:47.893+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:48.183 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:48.181+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:48.866 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:48.864+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:49.150 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:49.148+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:49.826 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:49.824+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:50.180 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:50.178+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:50.794 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:50.792+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:51.194 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:51.192+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:51.842 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:51.840+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:52.203 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:52.202+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:52.800 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:52.798+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:53.205 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:53.204+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:53.781 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:53.779+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:54.184 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:54.182+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:54.833 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:54.829+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:55.156 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:55.154+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:55.873 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:55.871+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:56.108 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:56.106+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:56.882 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:56.880+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:57.135 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:57.134+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:57.846 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:57.844+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:58.168 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:58.167+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:58.807 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:58.805+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:59.137 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:31:59.134+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:31:59.773 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:31:59.771+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:00.176 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:00.174+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:00.790 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:00.788+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:01.164 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:01.162+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:01.814 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:01.811+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:02.136 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:02.134+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:02.813 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:02.812+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:03.156 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:03.154+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:03.779 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:03.777+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:04.168 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:04.166+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:04.783 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:04.781+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:05.208 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:05.206+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:05.808 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:05.806+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:06.172 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:06.170+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:06.763 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:06.761+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:07.146 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:07.144+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:07.722 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:07.720+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:08.137 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:08.135+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:08.686 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:08.684+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:09.133 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:09.131+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:09.681 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:09.679+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:10.162 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:10.160+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:10.730 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:10.728+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:11.117 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:11.115+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:11.764 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:11.762+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:12.073 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:12.071+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:12.720 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:12.718+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:13.095 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:13.093+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:13.754 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:13.752+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:14.135 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:14.134+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:14.731 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:14.729+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:15.115 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:15.113+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:15.728 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:15.726+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:16.092 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:16.090+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:16.759 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:16.757+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:17.058 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:17.056+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:17.751 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:17.749+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:18.032 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:18.030+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:18.733 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:18.731+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:19.009 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:19.007+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:19.736 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:19.734+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:20.004 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:20.002+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:20.761 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:20.759+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:20.993 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:20.992+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:21.794 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:21.793+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:22.040 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:22.038+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:22.757 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:22.755+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:23.019 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:23.017+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:23.735 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:23.734+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:24.019 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:24.018+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:24.758 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:24.757+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:24.978 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:24.977+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:25.805 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:25.804+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:25.954 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:25.953+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:26.855 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:26.854+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:26.985 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:26.984+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:27.893 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:27.892+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:27.951 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:27.950+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:28.878 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:28.877+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:28.928 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:28.927+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:29.833 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:29.833+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:29.960 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:29.959+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:30.799 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:30.798+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:30.948 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:30.947+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:31.818 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:31.817+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:31.943 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:31.943+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:32.864 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:32.863+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:32.926 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:32.924+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:33.818 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:33.817+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:33.945 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:33.944+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:34.810 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:34.809+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:34.957 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:34.956+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:35.858 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:35.857+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:35.962 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:35.962+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:36.858 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:36.857+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:36.926 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:36.925+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:37.901 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:37.900+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:37.972 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:37.971+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:38.946 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:38.945+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:38.964 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:38.963+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:39.947 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:39.947+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:39.989 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:39.988+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:40.940 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:40.939+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:40.978 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:40.977+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:41.967 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:41.966+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:42.012 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:42.011+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:42.926 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:42.925+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:43.032 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:43.031+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:43.937 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:43.935+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:43.992 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:43.990+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:44.961 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:44.959+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:44.998 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:44.996+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:45.971 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:45.970+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:45.990 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:45.989+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:46.964 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:46.962+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:46.975 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:46.973+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:47.922 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:47.920+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:47.989 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:47.987+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:48.925 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:48.923+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:48.969 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:48.968+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:49.916 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:49.914+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:49.969 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:49.967+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:50.945 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:50.943+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:50.977 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:50.975+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:51.946 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:51.944+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:51.981 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:51.980+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:52.991 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:52.990+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:52.994 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:52.992+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:53.967 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:53.966+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:53.989 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:53.988+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:54.934 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:54.932+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:54.987 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:54.985+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:55.960 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:55.958+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:55.974 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:55.972+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:56.998 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:56.996+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:57.006 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:57.004+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:57.974 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:57.972+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:58.036 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:58.034+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:58.967 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:58.965+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:59.062 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:32:59.060+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:32:59.969 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:32:59.967+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:00.018 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:00.016+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:00.946 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:00.944+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:00.988 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:00.986+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:01.899 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:01.898+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:01.961 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:01.959+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:02.925 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:02.923+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:02.970 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:02.968+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:03.922 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:03.920+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:03.983 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:03.982+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:04.952 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:04.950+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:04.981 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:04.979+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:05.924 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:05.922+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:05.952 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:05.950+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:06.883 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:06.881+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:06.963 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:06.961+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:07.885 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:07.884+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:07.997 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:07.995+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:08.887 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:08.885+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:09.005 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:09.003+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:09.920 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:09.919+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:09.971 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:09.969+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:10.928 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:10.927+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:10.945 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:10.943+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:11.898 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:11.896+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:11.914 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:11.911+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:12.869 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:12.867+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:12.940 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:12.938+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:13.859 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:13.858+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:13.962 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:13.961+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:14.834 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:14.832+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:14.958 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:14.956+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:15.839 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:15.837+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:15.942 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:15.940+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:16.842 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:16.840+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:16.989 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:16.988+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:17.832 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:17.830+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:17.990 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:17.988+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:18.795 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:18.793+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:18.942 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:18.940+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:19.789 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:19.787+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:19.927 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:19.925+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:20.796 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:20.794+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:20.935 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:20.933+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:21.800 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:21.798+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:21.965 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:21.964+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:22.799 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:22.798+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:22.922 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:22.922+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:23.824 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:23.824+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:23.966 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:23.965+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:24.813 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:24.812+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:24.938 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:24.937+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:25.834 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:25.834+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:25.942 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:25.942+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:26.841 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:26.840+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:26.949 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:26.948+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:27.806 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:27.805+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:27.960 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:27.959+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:28.803 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:28.802+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:28.921 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:28.921+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:29.842 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:29.841+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:29.937 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:29.936+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:30.878 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:30.877+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:30.942 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:30.942+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:31.863 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:31.862+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:31.954 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:31.954+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:32.842 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:32.842+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:32.909 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:32.908+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:33.876 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:33.875+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:33.896 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:33.895+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:34.873 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:34.872+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:34.905 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:34.904+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:35.861 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:35.860+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:35.929 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:35.928+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:36.820 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:36.819+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:36.882 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:36.881+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:37.783 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:37.783+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:37.835 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:37.835+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:38.760 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:38.760+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:38.866 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:38.866+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:39.771 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:39.770+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:39.840 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:39.839+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:40.757 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:40.756+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:40.819 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:40.819+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:41.754 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:41.753+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:41.810 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:41.810+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:42.804 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:42.802+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:42.856 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:42.854+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:43.768 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:43.766+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:43.905 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:43.903+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:44.763 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:44.761+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:44.884 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:44.883+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:45.756 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:45.754+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:45.900 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:45.898+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:46.727 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:46.725+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:46.874 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:46.872+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:47.687 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:47.685+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:47.855 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:47.853+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:48.671 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:48.669+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:48.900 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:48.898+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:49.683 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:49.681+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:49.876 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:49.874+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:50.641 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:50.639+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:50.851 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:50.849+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:51.679 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:51.677+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:51.854 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:51.852+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:52.679 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:52.677+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:52.824 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:52.822+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:53.679 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:53.677+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:53.869 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:53.868+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:54.641 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:54.640+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:54.833 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:54.831+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:55.681 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:55.680+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:55.814 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:55.812+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:56.669 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:56.667+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:56.778 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:56.776+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:57.661 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:57.659+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:57.779 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:57.777+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:58.693 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:58.692+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:58.758 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:58.757+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:59.652 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:33:59.650+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:33:59.758 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:33:59.757+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:00.670 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:00.669+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:00.798 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:00.797+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:01.622 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:01.620+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:01.791 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:01.789+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:02.621 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:02.619+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:02.774 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:02.772+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:03.660 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:03.658+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:03.780 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:03.778+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:04.672 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:04.670+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:04.777 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:04.775+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:05.700 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:05.698+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:05.751 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:05.749+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:06.677 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:06.676+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:06.755 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:06.753+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:07.697 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:07.695+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:07.738 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:07.737+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:08.694 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:08.692+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:08.724 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:08.723+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:09.683 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:09.681+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:09.761 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:09.759+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:10.716 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:10.715+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:10.732 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:10.731+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:11.718 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:11.716+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:11.744 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:11.743+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:12.683 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:12.681+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:12.759 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:12.757+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:13.716 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:13.714+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:13.752 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:13.751+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:14.735 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:14.733+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:14.735 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:14.733+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:15.701 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:15.700+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:15.704 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:15.703+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:16.742 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:16.740+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:16.745 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:16.744+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:17.729 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:17.727+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:17.762 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:17.761+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:18.729 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:18.727+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:18.789 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:18.788+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:19.752 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:19.750+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:19.791 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:19.790+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:20.731 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:20.730+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:20.746 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:20.745+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:21.757 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:21.757+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:21.775 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:21.774+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:22.736 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:22.735+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:22.785 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:22.785+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:23.776 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:23.775+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:23.781 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:23.781+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:24.746 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:24.745+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:24.797 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:24.796+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:25.729 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:25.728+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:25.846 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:25.845+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:26.741 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:26.741+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:26.889 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:26.888+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:27.743 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:27.742+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:27.938 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:27.937+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:28.702 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:28.701+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:28.942 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:28.941+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:29.705 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:29.704+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:29.898 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:29.898+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:30.701 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:30.700+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:30.941 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:30.941+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:31.673 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:31.672+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:31.974 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:31.973+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:32.672 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:32.671+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:32.995 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:32.995+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:33.674 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:33.673+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:33.972 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:33.971+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:34.662 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:34.661+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:34.939 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:34.939+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:35.705 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:35.705+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:35.918 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:35.917+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:36.736 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:36.735+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:36.906 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:36.905+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:37.749 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:37.748+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:37.948 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:37.948+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:38.739 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:38.739+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:38.902 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:38.901+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:39.738 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:39.737+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:39.920 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:39.919+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:40.715 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:40.713+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:40.911 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:40.909+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:41.674 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:41.672+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:41.922 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:41.920+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:42.676 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:42.674+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:42.889 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:42.887+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:43.627 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:43.625+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:43.843 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:43.842+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:44.600 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:44.598+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:44.805 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:44.804+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:45.648 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:45.646+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:45.781 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:45.779+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:46.653 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:46.650+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:46.823 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:46.821+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:47.662 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:47.660+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:47.858 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:47.856+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:48.667 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:48.665+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:48.856 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:48.854+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:49.657 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:49.655+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:49.859 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:49.858+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:50.613 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:50.612+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:50.812 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:50.810+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:51.655 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:51.653+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:51.777 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:51.775+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:52.636 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:52.634+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:52.817 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:52.815+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:53.635 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:53.633+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:53.774 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:53.772+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:54.598 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:54.597+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:54.740 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:54.738+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:55.646 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:55.644+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:55.786 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:55.784+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:56.672 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:56.670+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:56.822 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:56.820+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:57.710 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:57.708+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:57.812 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:57.810+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:58.676 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:58.674+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:58.807 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:58.805+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:59.646 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:34:59.644+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:34:59.790 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:34:59.788+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:00.695 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:00.693+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:00.795 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:00.793+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:01.703 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:01.700+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:01.828 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:01.826+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:02.706 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:02.704+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:02.815 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:02.813+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:03.681 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:03.679+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:03.792 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:03.791+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:04.712 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:04.710+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:04.816 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:04.814+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:05.742 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:05.738+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:05.819 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:05.818+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:06.694 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:06.692+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:06.836 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:06.834+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:07.711 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:07.709+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:07.829 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:07.827+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:08.668 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:08.666+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:08.828 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:08.826+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:09.688 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:09.686+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:09.814 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:09.812+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:10.695 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:10.693+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:10.822 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:10.820+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:11.706 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:11.704+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:11.869 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:11.867+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:12.719 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:12.717+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:12.908 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:12.906+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:13.743 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:13.742+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:13.936 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:13.934+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:14.748 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:14.746+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:14.914 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:14.913+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:15.764 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:15.763+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:15.897 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:15.896+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:16.775 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:16.773+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:16.858 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:16.856+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:17.739 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:17.737+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:17.817 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:17.815+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:18.752 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:18.751+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:18.863 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:18.863+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:19.756 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:19.755+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:19.882 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:19.881+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:20.804 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:20.803+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:20.840 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:20.839+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:21.791 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:21.790+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:21.888 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:21.887+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:22.761 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:22.760+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:22.918 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:22.917+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:23.757 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:23.756+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:23.935 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:23.934+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:24.750 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:24.750+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:24.911 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:24.910+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:25.775 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:25.774+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:25.898 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:25.897+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:26.744 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:26.743+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:26.885 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:26.884+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:27.740 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:27.739+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:27.841 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:27.840+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:28.731 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:28.730+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:28.809 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:28.808+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:29.713 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:29.712+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:29.779 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:29.778+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:30.675 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:30.674+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:30.814 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:30.813+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:31.707 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:31.706+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:31.810 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:31.809+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:32.715 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:32.715+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:32.840 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:32.840+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:33.669 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:33.668+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:33.812 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:33.811+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:34.686 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:34.685+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:34.818 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:34.817+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:35.689 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:35.689+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:35.798 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:35.797+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:36.669 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:36.668+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:36.834 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:36.833+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:37.705 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:37.704+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:37.844 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:37.843+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:38.710 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:38.709+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:38.891 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:38.889+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:39.719 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:39.717+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:39.905 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:39.904+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:40.725 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:40.724+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:40.869 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:40.867+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:41.714 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:41.712+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:41.827 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:41.825+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:42.763 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:42.762+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:42.827 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:42.825+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:43.758 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:43.756+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:43.813 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:43.811+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:44.733 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:44.731+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:44.814 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:44.812+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:45.772 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:45.771+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:45.844 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:45.842+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:46.739 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:46.737+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:46.884 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:46.881+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:47.740 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:47.738+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:47.869 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:47.867+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:48.708 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:48.706+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:48.825 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:48.823+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:49.743 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:49.742+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:49.814 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:49.812+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:50.727 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:50.725+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:50.857 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:50.855+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:51.681 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:51.680+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:51.845 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:51.844+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:52.637 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:52.636+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:52.892 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:52.890+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:53.646 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:53.644+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:53.927 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:53.925+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:54.609 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:54.607+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:54.954 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:54.952+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:55.569 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:55.567+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:55.917 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:55.915+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:56.615 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:56.613+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:56.932 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:56.931+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:57.587 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:57.585+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:57.893 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:57.892+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:58.562 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:58.560+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:58.905 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:58.903+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:59.523 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:35:59.521+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:35:59.902 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:35:59.900+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:00.491 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:00.490+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:00.929 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:00.927+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:01.469 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:01.468+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:01.890 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:01.888+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:02.519 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:02.518+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:02.852 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:02.850+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:03.535 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:03.533+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:03.879 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:03.877+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:04.545 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:04.543+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:04.852 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:04.850+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:05.517 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:05.515+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:05.902 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:05.900+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:06.484 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:06.482+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:06.907 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:06.905+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:07.492 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:07.490+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:07.914 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:07.912+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:08.474 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:08.472+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:08.911 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:08.909+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:09.426 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:09.424+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:09.904 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:09.902+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:10.421 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:10.419+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:10.953 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:10.952+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:11.467 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:11.465+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:11.970 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:11.969+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:12.466 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:12.464+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:12.958 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:12.956+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:13.516 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:13.514+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:13.981 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:13.979+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:14.507 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:14.506+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:14.986 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:14.985+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:15.539 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:15.537+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:16.014 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:16.013+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:16.577 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:16.576+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:17.054 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:17.053+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:17.562 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:17.562+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:18.080 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:18.079+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:18.548 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:18.547+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:19.086 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:19.085+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:19.551 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:19.551+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:20.124 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:20.123+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:20.561 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:20.561+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:21.161 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:21.160+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:21.606 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:21.605+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:22.174 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:22.173+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:22.562 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:22.561+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:23.127 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:23.126+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:23.572 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:23.571+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:24.107 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:24.106+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:24.567 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:24.566+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:25.067 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:25.066+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:25.529 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:25.529+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:26.114 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:26.113+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:26.484 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:26.483+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:27.091 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:27.091+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:27.521 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:27.521+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:28.126 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:28.125+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:28.481 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:28.481+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:29.174 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:29.173+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:29.487 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:29.487+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:30.177 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:30.176+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:30.463 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:30.463+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:31.148 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:31.147+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:31.439 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:31.439+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:32.177 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:32.176+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:32.452 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:32.451+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:33.156 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:33.156+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:33.501 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:33.500+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:34.185 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:34.185+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:34.539 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:34.538+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:35.198 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:35.197+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:35.561 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:35.560+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:36.231 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:36.230+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:36.564 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:36.563+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:37.271 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:37.270+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:37.584 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:37.582+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:38.294 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:38.292+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:38.576 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:38.574+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:39.325 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:39.323+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:39.582 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:39.581+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:40.357 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:40.355+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:40.589 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:40.587+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:41.404 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:41.402+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:41.611 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:41.609+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:42.410 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:42.408+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:42.583 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:42.581+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:43.413 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:43.411+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:43.575 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:43.573+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:44.373 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:44.372+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:44.579 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:44.577+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:45.381 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:45.380+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:45.562 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:45.560+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:46.429 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:46.427+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:46.571 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:46.569+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:47.388 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:47.386+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:47.582 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:47.580+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:48.375 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:48.373+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:48.612 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:48.611+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:49.387 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:49.385+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:49.584 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:49.582+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:50.389 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:50.387+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:50.622 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:50.620+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:51.383 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:51.381+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:51.641 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:51.640+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:52.414 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:52.412+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:52.595 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:52.594+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:53.366 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:53.363+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:53.564 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:53.563+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:54.349 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:54.347+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:54.566 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:54.564+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:55.365 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:55.363+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:55.552 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:55.550+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:56.327 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:56.325+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:56.543 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:56.541+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:57.279 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:57.277+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:57.582 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:57.580+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:58.295 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:58.293+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:58.607 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:58.606+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:59.274 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:36:59.272+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:36:59.561 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:36:59.559+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:00.242 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:00.240+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:00.517 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:00.515+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:01.290 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:01.288+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:01.539 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:01.537+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:02.242 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:02.240+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:02.555 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:02.553+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:03.219 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:03.218+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:03.593 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:03.591+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:04.253 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:04.252+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:04.605 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:04.603+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:05.258 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:05.256+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:05.596 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:05.594+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:06.219 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:06.217+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:06.608 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:06.607+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:07.259 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:07.257+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:07.624 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:07.622+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:08.246 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:08.244+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:08.598 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:08.596+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:09.282 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:09.280+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:09.556 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:09.554+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:10.247 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:10.245+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:10.590 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:10.588+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:11.268 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:11.266+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:11.611 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:11.609+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:12.244 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:12.242+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:12.634 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:12.632+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:13.252 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:13.250+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:13.668 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:13.666+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:14.276 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:14.275+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:14.641 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:14.640+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:15.312 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:15.310+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:15.668 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:15.667+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:16.310 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:16.309+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:16.630 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:16.629+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:17.331 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:17.330+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:17.591 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:17.591+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:18.330 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:18.329+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:18.603 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:18.602+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:19.303 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:19.302+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:19.579 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:19.578+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:20.277 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:20.277+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:20.580 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:20.580+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:21.243 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:21.242+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:21.607 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:21.606+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:22.212 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:22.211+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:22.647 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:22.646+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:23.183 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:23.182+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:23.613 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:23.613+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:24.174 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:24.173+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:24.629 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:24.629+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:25.180 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:25.180+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:25.591 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:25.590+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:26.158 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:26.158+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:26.556 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:26.555+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:27.174 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:27.173+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:27.510 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:27.509+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:28.128 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:28.128+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:28.549 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:28.548+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:29.095 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:29.094+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:29.504 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:29.503+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:30.055 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:30.054+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:30.538 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:30.537+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:31.047 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:31.047+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:31.569 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:31.568+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:32.090 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:32.089+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:32.566 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:32.565+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:33.123 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:33.122+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:33.604 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:33.603+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:34.079 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:34.077+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:34.630 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:34.630+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:35.095 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:35.094+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:35.629 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:35.627+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:36.092 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:36.091+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:36.659 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:36.657+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:37.089 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:37.087+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:37.649 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:37.647+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:38.080 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:38.079+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:38.628 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:38.626+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:39.093 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:39.091+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:39.604 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:39.602+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:40.128 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:40.126+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:40.597 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:40.595+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:41.111 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:41.109+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:41.636 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:41.634+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:42.120 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:42.119+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:42.656 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:42.654+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:43.095 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:43.093+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:43.616 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:43.615+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:44.073 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:44.071+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:44.616 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:44.615+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:45.043 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:45.041+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:45.663 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:45.661+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:46.075 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:46.074+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:46.682 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:46.681+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:47.052 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:47.051+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:47.636 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:47.634+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:48.036 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:48.034+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:48.640 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:48.639+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:49.028 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:49.026+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:49.691 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:49.689+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:50.074 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:50.072+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:50.672 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:50.670+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:51.073 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:51.071+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:51.688 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:51.687+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:52.044 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:52.043+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:52.671 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:52.669+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:53.084 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:53.083+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:53.669 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:53.667+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:54.098 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:54.096+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:54.638 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:54.637+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:55.110 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:55.109+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:55.650 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:55.648+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:56.107 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:56.105+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:56.680 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:56.679+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:57.121 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:57.119+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:57.682 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:57.680+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:58.081 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:58.079+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:58.687 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:58.685+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:59.101 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:37:59.099+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:37:59.705 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:37:59.703+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:00.147 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:00.145+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:00.676 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:00.674+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:01.122 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:01.120+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:01.698 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:01.696+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:02.141 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:02.139+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:02.668 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:02.667+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:03.103 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:03.101+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:03.679 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:03.677+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:04.064 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:04.062+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:04.654 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:04.652+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:05.090 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:05.089+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:05.646 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:05.644+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:06.048 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:06.046+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:06.600 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:06.598+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:07.099 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:07.097+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:07.607 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:07.605+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:08.148 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:08.146+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:08.609 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:08.607+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:09.138 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:09.137+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:09.595 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:09.592+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:10.184 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:10.182+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:10.578 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:10.576+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:11.162 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:11.160+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:11.591 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:11.589+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:12.209 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:12.207+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:12.562 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:12.561+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:13.200 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:13.198+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:13.519 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:13.518+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:14.208 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:14.207+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:14.560 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:14.559+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:15.198 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:15.198+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:15.556 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:15.556+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:16.184 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:16.183+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:16.512 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:16.511+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:17.139 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:17.138+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:17.486 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:17.486+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:18.173 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:18.172+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:18.472 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:18.471+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:19.212 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:19.211+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:19.452 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:19.451+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:20.246 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:20.246+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:20.444 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:20.443+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:21.220 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:21.219+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:21.479 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:21.478+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:22.230 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:22.229+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:22.509 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:22.508+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:23.186 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:23.185+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:23.531 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:23.531+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:24.138 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:24.137+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:24.542 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:24.541+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:25.138 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:25.137+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:25.502 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:25.501+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:26.166 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:26.165+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:26.455 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:26.454+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:27.183 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:27.182+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:27.473 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:27.472+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:28.218 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:28.217+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:28.442 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:28.441+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:29.194 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:29.193+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:29.419 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:29.418+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:30.226 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:30.225+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:30.433 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:30.432+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:31.267 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:31.266+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:31.468 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:31.468+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:32.253 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:32.252+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:32.492 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:32.492+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:33.275 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:33.275+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:33.521 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:33.518+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:34.277 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:34.276+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:34.562 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:34.560+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:35.281 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:35.280+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:35.528 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:35.526+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:36.312 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:36.310+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:36.517 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:36.516+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:37.272 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:37.270+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:37.556 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:37.554+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:38.273 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:38.270+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:38.508 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:38.506+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:39.284 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:39.283+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:39.472 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:39.471+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:40.292 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:40.291+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:40.427 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:40.425+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:41.259 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:41.257+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:41.438 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:41.436+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:42.304 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:42.302+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:42.453 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:42.452+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:43.273 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:43.271+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:43.426 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:43.425+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:44.272 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:44.270+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:44.435 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:44.434+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:45.263 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:45.262+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:45.475 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:45.473+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:46.220 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:46.218+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:46.460 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:46.458+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:47.218 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:47.216+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:47.488 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:47.486+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:48.254 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:48.253+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:48.439 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:48.437+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:49.299 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:49.297+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:49.415 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:49.414+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:50.297 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:50.296+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:50.446 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:50.444+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:51.258 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:51.256+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:51.490 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:51.488+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:52.266 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:52.265+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:52.500 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:52.498+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:53.280 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:53.278+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:53.517 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:53.515+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:54.233 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:54.232+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:54.501 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:54.499+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:55.199 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:55.197+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:55.510 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:55.509+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:56.228 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:56.226+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:56.523 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:56.521+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:57.269 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:57.267+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:57.481 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:57.479+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:58.245 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:58.243+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:58.519 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:58.517+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:59.208 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:38:59.206+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:38:59.488 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:38:59.487+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:00.180 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:00.178+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:00.488 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:00.486+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:01.205 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:01.204+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:01.483 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:01.482+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:02.190 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:02.188+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:02.506 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:02.504+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:03.173 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:03.171+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:03.475 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:03.474+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:04.197 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:04.196+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:04.458 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:04.456+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:05.163 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:05.162+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:05.446 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:05.444+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:06.167 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:06.165+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:06.440 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:06.438+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:07.172 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:07.170+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:07.489 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:07.487+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:08.158 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:08.156+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:08.446 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:08.444+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:09.130 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:09.128+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:09.477 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:09.476+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:10.100 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:10.098+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:10.505 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:10.503+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:11.066 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:11.064+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:11.481 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:11.480+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:12.048 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:12.048+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:12.488 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:12.487+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:13.042 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:13.042+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:13.499 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:13.498+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:14.063 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:14.062+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:14.470 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:14.469+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:15.083 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:15.082+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:15.444 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:15.443+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:16.082 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:16.081+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:16.486 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:16.486+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:17.065 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:17.064+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:17.516 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:17.516+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:18.039 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:18.038+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:18.533 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:18.530+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:19.023 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:19.023+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:19.530 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:19.529+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:20.049 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:20.048+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:20.505 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:20.504+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:21.095 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:21.094+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:21.458 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:21.457+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:22.062 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:22.062+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:22.448 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:22.448+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:23.070 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:23.069+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:23.462 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:23.461+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:24.046 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:24.046+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:24.425 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:24.424+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:25.036 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:25.035+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:25.410 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:25.409+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:25.997 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:25.996+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:26.386 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:26.386+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:26.976 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:26.975+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:27.416 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:27.415+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:27.938 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:27.938+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:28.389 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:28.388+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:28.915 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:28.914+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:29.342 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:29.341+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:29.871 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:29.870+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:30.295 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:30.295+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:30.851 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:30.850+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:31.326 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:31.325+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:31.897 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:31.896+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:32.351 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:32.349+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:32.908 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:32.907+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:33.327 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:33.325+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:33.881 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:33.880+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:34.374 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:34.373+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:34.919 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:34.917+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:35.394 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:35.393+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:35.924 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:35.922+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:36.390 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:36.388+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:36.924 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:36.922+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:37.424 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:37.422+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:37.888 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:37.887+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:38.459 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:38.457+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:38.862 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:38.860+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:39.488 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:39.486+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:39.814 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:39.813+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:40.538 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:40.536+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:40.851 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:40.849+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:41.585 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:41.583+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:41.819 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:41.818+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:42.570 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:42.569+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:42.793 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:42.791+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:43.527 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:43.526+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:43.815 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:43.813+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:44.558 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:44.557+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:44.799 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:44.797+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:45.594 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:45.593+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:45.793 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:45.791+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:46.545 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:46.544+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:46.816 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:46.814+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:47.546 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:47.545+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:47.803 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:47.801+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:48.524 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:48.522+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:48.845 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:48.843+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:49.532 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:49.530+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:49.854 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:49.852+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:50.499 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:50.497+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:50.892 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:50.891+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:51.456 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:51.454+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:51.907 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:51.905+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:52.461 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:52.459+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:52.890 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:52.889+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:53.463 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:53.461+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:53.929 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:53.927+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:54.509 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:54.507+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:54.930 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:54.929+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:55.525 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:55.523+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:55.962 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:55.961+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:56.535 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:56.533+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:56.930 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:56.928+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:57.546 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:57.544+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:57.940 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:57.938+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:58.523 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:58.521+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:58.909 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:58.908+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:59.503 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:39:59.501+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:39:59.890 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:39:59.888+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:00.483 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:00.481+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:00.895 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:00.894+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:01.438 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:01.436+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:01.914 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:01.913+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:02.424 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:02.423+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:02.903 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:02.901+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:03.426 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:03.424+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:03.916 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:03.914+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:04.434 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:04.432+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:04.934 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:04.933+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:05.410 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:05.408+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:05.965 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:05.964+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:06.425 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:06.423+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:06.946 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:06.944+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:07.378 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:07.377+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:07.930 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:07.929+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:08.391 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:08.389+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:08.904 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:08.902+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:09.398 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:09.396+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:09.904 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:09.902+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:10.369 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:10.368+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:10.900 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:10.899+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:11.381 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:11.381+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:11.914 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:11.913+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:12.352 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:12.352+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:12.906 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:12.905+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:13.319 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:13.319+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:13.873 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:13.873+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:14.302 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:14.301+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:14.883 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:14.882+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:15.332 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:15.331+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:15.915 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:15.914+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:16.365 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:16.364+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:16.878 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:16.878+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:17.345 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:17.344+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:17.872 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:17.872+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:18.306 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:18.305+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:18.910 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:18.909+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:19.350 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:19.349+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:19.887 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:19.886+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:20.354 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:20.353+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:20.862 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:20.862+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:21.396 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:21.396+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:21.909 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:21.908+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:22.360 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:22.359+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:22.947 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:22.946+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:23.341 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:23.340+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:23.947 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:23.946+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:24.297 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:24.296+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:24.903 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:24.902+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:25.268 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:25.267+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:25.868 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:25.868+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:26.298 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:26.298+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:26.852 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:26.851+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:27.344 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:27.344+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:27.848 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:27.847+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:28.296 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:28.295+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:28.804 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:28.803+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:29.284 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:29.284+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:29.838 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:29.837+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:30.303 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:30.302+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:30.866 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:30.864+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:31.284 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:31.283+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:31.869 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:31.868+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:32.318 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:32.316+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:32.844 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:32.842+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:33.364 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:33.363+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:33.882 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:33.881+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:34.393 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:34.391+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:34.904 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:34.902+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:35.418 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:35.416+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:35.918 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:35.915+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:36.414 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:36.412+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:36.903 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:36.901+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:37.460 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:37.459+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:37.888 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:37.886+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:38.418 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:38.417+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:38.866 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:38.865+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:39.371 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:39.370+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:39.858 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:39.856+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:40.327 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:40.326+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:40.811 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:40.809+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:41.325 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:41.324+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:41.862 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:41.860+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:42.361 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:42.359+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:42.853 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:42.851+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:43.341 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:43.339+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:43.886 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:43.884+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:44.311 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:44.309+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:44.916 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:44.915+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:45.272 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:45.270+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:45.936 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:45.934+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:46.266 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:46.264+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:46.935 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:46.933+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:47.231 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:47.229+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:47.909 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:47.907+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:48.270 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:48.268+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:48.880 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:48.879+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:49.312 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:49.310+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:49.905 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:49.904+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:50.352 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:50.350+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:50.929 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:50.927+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:51.373 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:51.372+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:51.931 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:51.929+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:52.366 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:52.365+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:52.938 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:52.937+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:53.350 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:53.348+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:53.981 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:53.979+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:54.377 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:54.375+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:54.976 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:54.974+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:55.376 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:55.374+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:55.958 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:55.956+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:56.347 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:56.346+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:56.953 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:56.951+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:57.330 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:57.329+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:57.951 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:57.950+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:58.364 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:58.363+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:58.960 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:58.959+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:59.399 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:40:59.398+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:40:59.914 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:40:59.912+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:00.421 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:00.419+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:00.929 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:00.927+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:01.432 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:01.430+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:01.900 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:01.899+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:02.393 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:02.391+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:02.934 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:02.932+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:03.344 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:03.343+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:03.960 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:03.958+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:04.373 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:04.371+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:04.984 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:04.983+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:05.401 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:05.399+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:05.992 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:05.990+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:06.413 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:06.411+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:06.974 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:06.972+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:07.435 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:07.434+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:08.003 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:08.001+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:08.399 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:08.398+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:09.046 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:09.046+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:09.437 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:09.437+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:10.024 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:10.023+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:10.428 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:10.428+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:11.038 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:11.035+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:11.420 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:11.419+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:12.080 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:12.079+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:12.469 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:12.468+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:13.049 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:13.048+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:13.442 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:13.441+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:14.018 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:14.017+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:14.458 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:14.458+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:15.043 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:15.042+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:15.507 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:15.506+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:16.083 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:16.082+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:16.537 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:16.537+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:17.094 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:17.093+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:17.490 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:17.489+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:18.124 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:18.123+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:18.479 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:18.479+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:19.120 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:19.119+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:19.486 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:19.485+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:20.105 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:20.104+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:20.490 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:20.489+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:21.143 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:21.142+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:21.510 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:21.509+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:22.104 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:22.103+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:22.510 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:22.509+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:23.152 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:23.151+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:23.501 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:23.500+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:24.197 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:24.197+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:24.486 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:24.486+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:25.228 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:25.227+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:25.483 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:25.482+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:26.271 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:26.270+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:26.496 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:26.495+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:27.318 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:27.318+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:27.524 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:27.523+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:28.339 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:28.338+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:28.556 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:28.555+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:28.736 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T23:41:28.735+0000 7f994396e700 0 -- 172.21.15.167:0/1156974075 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x55b130f38ff0 msgr2=0x7f99280a6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-05T23:41:29.293 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:29.291+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:29.585 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:29.583+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:30.284 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:30.283+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:30.552 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:30.550+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:31.332 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:31.330+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:31.553 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:31.551+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:32.368 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:32.366+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:32.551 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:32.549+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:33.388 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:33.386+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:33.554 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:33.552+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:34.382 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:34.380+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:34.559 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:34.557+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:35.422 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:35.420+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:35.523 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:35.522+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:36.428 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:36.426+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:36.515 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:36.513+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:37.387 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:37.385+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:37.483 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:37.481+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:38.410 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:38.409+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:38.464 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:38.462+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:39.413 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:39.411+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:39.480 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:39.478+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:40.427 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:40.425+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:40.448 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:40.446+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:41.447 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:41.446+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:41.477 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:41.475+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:42.429 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:42.427+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:42.449 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:42.447+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:43.413 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:43.412+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:43.427 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:43.426+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:44.381 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:44.379+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:44.398 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:44.396+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:45.412 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:45.410+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:45.441 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:45.439+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:46.367 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:46.366+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:46.396 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:46.394+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:47.320 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:47.318+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:47.385 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:47.384+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:48.278 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:48.276+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:48.365 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:48.364+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:49.328 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:49.326+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:49.392 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:49.391+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:50.319 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:50.318+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:50.356 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:50.354+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:51.337 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:51.335+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:51.359 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:51.358+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:52.324 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:52.322+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:52.405 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:52.403+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:53.336 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:53.334+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:53.405 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:53.404+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:54.299 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:54.297+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:54.403 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:54.401+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:55.345 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:55.344+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:55.357 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:55.355+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:56.314 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:56.313+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:56.397 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:56.396+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:57.304 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:57.302+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:57.430 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:57.428+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:58.302 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:58.300+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:58.457 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:58.455+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:59.343 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:41:59.342+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:41:59.457 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:41:59.455+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:00.346 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:00.344+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:00.455 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:00.454+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:01.393 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:01.391+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:01.436 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:01.435+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:02.361 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:02.360+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:02.407 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:02.405+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:03.371 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:03.369+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:03.403 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:03.402+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:04.368 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:04.366+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:04.388 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:04.387+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:05.348 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:05.346+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:05.385 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:05.384+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:06.358 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:06.357+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:06.388 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:06.386+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:07.322 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:07.321+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:07.433 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:07.432+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:08.335 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:08.334+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:08.406 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:08.405+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:09.344 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:09.343+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:09.426 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:09.426+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:10.369 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:10.368+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:10.417 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:10.416+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:11.346 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:11.345+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:11.396 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:11.396+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:12.359 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:12.358+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:12.362 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:12.361+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:13.331 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:13.330+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:13.392 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:13.391+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:14.341 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:14.340+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:14.354 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:14.353+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:15.339 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:15.338+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:15.376 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:15.375+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:16.352 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:16.351+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:16.364 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:16.363+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:17.370 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:17.369+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:17.374 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:17.374+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:18.353 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:18.352+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:18.388 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:18.387+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:19.309 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:19.308+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:19.427 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:19.426+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:20.303 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:20.302+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:20.406 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:20.406+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:21.255 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:21.254+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:21.432 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:21.431+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:22.252 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:22.251+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:22.434 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:22.434+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:23.229 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:23.228+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:23.424 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:23.423+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:24.189 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:24.189+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:24.466 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:24.465+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:25.207 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:25.206+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:25.471 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:25.471+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:26.225 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:26.224+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:26.509 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:26.508+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:27.182 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:27.181+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:27.530 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:27.529+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:28.148 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:28.147+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:28.535 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:28.533+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:29.195 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:29.193+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:29.534 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:29.532+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:30.217 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:30.216+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:30.517 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:30.516+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:31.204 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:31.202+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:31.531 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:31.529+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:32.242 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:32.240+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:32.559 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:32.558+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:33.231 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:33.229+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:33.546 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:33.544+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:34.185 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:34.184+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:34.512 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:34.510+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:35.168 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:35.166+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:35.515 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:35.513+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:36.162 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:36.160+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:36.559 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:36.557+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:37.153 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:37.151+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:37.582 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:37.581+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:38.194 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:38.193+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:38.617 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:38.615+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:39.232 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:39.230+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:39.630 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:39.628+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:40.209 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:40.207+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:40.660 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:40.658+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:41.242 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:41.240+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:41.629 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:41.627+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:42.242 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:42.240+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:42.594 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:42.592+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:43.285 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:43.283+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:43.557 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:43.556+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:44.254 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:44.253+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:44.548 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:44.546+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:45.218 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:45.217+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:45.539 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:45.537+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:46.183 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:46.181+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:46.512 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:46.510+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:47.145 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:47.143+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:47.537 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:47.535+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:48.180 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:48.178+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:48.579 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:48.577+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:49.154 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:49.151+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:49.602 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:49.600+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:50.179 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:50.178+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:50.564 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:50.562+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:51.228 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:51.226+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:51.593 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:51.591+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:52.261 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:52.259+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:52.544 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:52.543+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:53.246 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:53.245+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:53.534 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:53.532+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:54.277 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:54.275+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:54.570 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:54.569+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:55.274 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:55.272+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:55.536 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:55.534+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:56.247 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:56.245+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:56.492 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:56.491+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:57.289 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:57.287+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:57.536 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:57.534+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:58.295 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:58.294+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:58.509 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:58.507+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:59.281 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:42:59.279+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:42:59.549 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:42:59.547+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:00.326 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:00.324+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:00.562 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:00.560+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:01.343 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:01.341+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:01.520 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:01.519+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:02.314 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:02.312+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:02.497 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:02.495+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:03.304 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:03.303+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:03.528 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:03.527+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:04.326 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:04.325+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:04.504 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:04.503+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:05.305 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:05.304+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:05.461 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:05.460+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:06.350 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:06.350+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:06.458 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:06.458+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:07.306 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:07.306+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:07.449 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:07.448+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:08.319 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:08.318+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:08.452 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:08.452+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:09.329 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:09.329+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:09.413 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:09.413+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:10.355 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:10.354+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:10.444 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:10.443+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:11.317 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:11.316+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:11.395 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:11.395+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:12.367 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:12.367+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:12.445 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:12.445+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:13.345 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:13.344+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:13.442 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:13.441+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:14.354 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:14.353+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:14.478 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:14.477+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:15.399 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:15.398+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:15.436 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:15.435+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:16.376 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:16.376+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:16.464 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:16.464+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:17.405 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:17.404+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:17.463 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:17.463+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:18.429 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:18.428+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:18.431 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:18.430+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:19.390 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:19.389+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:19.461 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:19.458+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:20.394 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:20.394+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:20.413 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:20.413+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:21.434 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:21.434+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:21.445 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:21.444+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:22.435 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:22.435+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:22.466 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:22.465+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:23.396 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:23.396+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:23.431 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:23.431+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:24.386 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:24.386+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:24.392 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:24.392+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:25.348 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:25.347+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:25.379 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:25.379+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:26.345 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:26.343+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:26.377 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:26.375+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:27.375 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:27.373+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:27.406 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:27.404+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:28.344 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:28.342+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:28.453 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:28.451+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:29.349 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:29.347+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:29.405 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:29.403+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:30.383 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:30.381+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:30.426 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:30.425+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:31.357 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:31.355+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:31.391 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:31.390+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:32.343 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:32.341+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:32.394 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:32.393+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:33.361 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:33.359+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:33.393 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:33.391+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:34.405 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:34.403+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:34.414 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:34.413+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:35.415 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:35.414+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:35.451 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:35.450+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:36.396 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:36.394+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:36.425 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:36.424+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:37.437 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:37.435+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:37.459 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:37.458+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:38.398 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:38.396+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:38.475 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:38.473+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:39.435 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:39.433+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:39.510 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:39.508+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:40.443 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:40.442+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:40.479 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:40.477+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:41.411 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:41.409+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:41.440 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:41.438+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:42.405 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:42.403+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:42.450 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:42.448+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:43.430 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:43.428+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:43.435 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:43.433+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:44.401 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:44.399+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:44.462 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:44.460+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:45.417 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:45.415+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:45.442 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:45.440+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:46.389 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:46.387+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:46.461 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:46.459+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:47.407 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:47.406+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:47.496 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:47.494+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:48.446 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:48.444+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:48.473 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:48.471+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:49.407 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:49.405+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:49.445 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:49.444+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:50.428 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:50.426+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:50.435 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:50.433+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:51.406 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:51.405+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:51.471 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:51.469+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:52.368 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:52.367+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:52.475 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:52.474+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:53.345 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:53.342+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:53.461 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:53.459+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:54.376 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:54.375+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:54.484 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:54.483+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:55.425 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:55.423+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:55.506 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:55.504+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:56.396 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:56.394+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:56.543 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:56.542+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:57.415 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:57.413+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:57.536 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:57.534+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:58.376 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:58.374+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:58.542 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:58.540+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:59.357 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:43:59.355+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:43:59.543 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:43:59.541+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:00.329 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:00.327+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:00.562 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:00.560+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:01.350 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:01.349+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:01.611 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:01.610+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:02.381 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:02.380+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:02.645 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:02.644+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:03.343 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:03.341+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:03.663 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:03.662+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:04.357 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:04.355+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:04.675 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:04.675+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:05.351 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:05.350+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:05.681 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:05.680+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:06.366 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:06.365+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:06.729 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:06.728+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:07.397 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:07.397+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:07.748 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:07.748+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:08.441 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:08.440+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:08.724 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:08.723+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:09.439 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:09.438+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:09.722 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:09.722+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:10.417 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:10.417+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:10.683 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:10.682+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:11.450 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:11.449+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:11.734 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:11.733+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:12.462 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:12.462+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:12.764 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:12.763+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:13.475 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:13.475+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:13.794 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:13.794+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:14.439 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:14.438+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:14.774 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:14.773+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:15.425 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:15.422+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:15.795 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:15.794+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:16.397 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:16.397+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:16.815 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:16.814+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:17.367 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:17.366+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:17.769 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:17.768+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:18.413 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:18.412+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:18.745 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:18.744+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:19.416 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:19.415+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:19.701 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:19.700+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:20.429 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:20.429+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:20.694 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:20.693+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:21.461 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:21.461+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:21.676 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:21.675+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:22.495 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:22.495+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:22.718 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:22.717+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:23.475 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:23.474+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:23.752 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:23.751+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:24.433 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:24.432+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:24.771 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:24.769+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:25.405 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:25.403+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:25.729 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:25.728+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:26.382 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:26.381+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:26.704 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:26.702+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:27.357 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:27.356+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:27.700 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:27.698+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:28.345 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:28.344+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:28.667 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:28.666+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:29.350 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:29.348+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:29.702 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:29.700+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:30.400 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:30.399+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:30.712 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:30.711+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:31.443 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:31.441+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:31.759 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:31.757+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:32.479 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:32.478+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:32.719 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:32.718+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:33.483 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:33.482+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:33.740 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:33.739+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:34.452 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:34.450+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:34.712 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:34.710+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:35.438 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:35.436+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:35.753 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:35.752+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:36.402 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:36.400+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:36.761 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:36.760+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:37.372 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:37.371+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:37.774 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:37.772+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:38.342 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:38.340+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:38.780 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:38.778+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:39.306 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:39.304+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:39.749 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:39.747+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:40.258 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:40.256+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:40.770 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:40.768+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:41.306 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:41.304+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:41.768 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:41.767+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:42.301 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:42.299+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:42.724 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:42.722+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:43.259 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:43.258+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:43.718 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:43.716+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:44.298 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:44.296+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:44.700 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:44.698+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:45.275 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:45.273+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:45.699 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:45.697+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:46.305 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:46.303+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:46.715 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:46.714+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:47.295 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:47.294+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:47.765 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:47.764+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:48.314 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:48.313+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:48.740 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:48.738+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:49.297 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:49.296+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:49.753 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:49.751+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:50.279 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:50.277+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:50.739 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:50.738+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:51.285 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:51.283+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:51.705 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:51.703+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:52.296 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:52.294+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:52.727 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:52.725+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:53.329 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:53.327+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:53.700 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:53.698+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:54.294 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:54.293+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:54.707 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:54.706+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:55.302 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:55.300+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:55.669 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:55.667+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:56.332 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:56.330+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:56.633 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:56.631+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:57.354 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:57.353+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:57.625 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:57.623+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:58.363 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:58.361+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:58.629 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:58.628+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:59.413 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:44:59.412+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:44:59.598 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:44:59.596+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:00.404 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:00.403+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:00.619 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:00.618+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:01.444 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:01.442+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:01.598 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:01.597+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:02.408 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:02.408+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:02.627 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:02.626+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:03.443 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:03.443+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:03.660 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:03.660+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:04.447 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:04.446+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:04.639 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:04.638+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:05.485 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:05.485+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:05.597 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:05.597+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:06.498 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:06.498+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:06.618 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:06.618+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:07.507 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:07.507+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:07.629 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:07.629+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:08.506 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:08.505+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:08.614 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:08.613+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:09.522 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:09.522+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:09.572 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:09.571+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:10.502 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:10.501+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:10.620 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:10.620+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:11.464 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:11.463+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:11.584 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:11.584+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:12.483 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:12.482+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:12.622 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:12.621+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:13.454 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:13.454+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:13.606 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:13.605+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:14.475 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:14.475+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:14.561 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:14.561+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:15.452 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:15.452+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:15.592 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:15.591+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:16.466 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:16.465+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:16.548 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:16.547+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:17.432 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:17.431+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:17.597 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:17.596+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:18.422 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:18.422+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:18.569 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:18.568+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:19.463 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:19.462+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:19.556 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:19.556+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:20.491 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:20.490+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:20.569 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:20.569+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:21.490 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:21.490+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:21.538 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:21.538+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:22.444 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:22.442+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:22.574 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:22.572+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:23.395 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:23.393+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:23.595 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:23.594+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:24.432 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:24.431+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:24.600 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:24.598+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:25.391 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:25.389+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:25.595 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:25.593+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:26.437 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:26.435+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:26.594 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:26.592+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:27.466 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:27.464+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:27.634 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:27.633+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:28.431 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:28.429+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:28.627 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:28.625+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:29.384 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:29.382+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:29.659 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:29.658+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:30.344 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:30.342+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:30.638 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:30.636+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:31.387 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:31.385+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:31.622 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:31.620+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:32.383 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:32.382+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:32.582 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:32.580+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:33.383 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:33.382+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:33.553 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:33.552+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:34.358 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:34.356+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:34.512 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:34.510+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:35.399 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:35.398+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:35.482 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:35.481+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:36.374 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:36.372+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:36.509 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:36.507+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:37.405 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:37.404+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:37.529 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:37.527+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:38.426 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:38.424+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:38.512 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:38.510+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:39.425 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:39.424+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:39.549 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:39.547+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:40.471 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:40.470+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:40.506 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:40.504+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:41.490 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:41.488+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:41.518 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:41.516+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:42.470 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:42.468+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:42.558 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:42.556+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:43.479 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:43.477+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:43.589 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:43.587+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:44.446 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:44.444+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:44.607 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:44.605+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:45.492 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:45.490+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:45.599 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:45.598+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:46.485 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:46.484+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:46.645 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:46.643+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:47.520 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:47.519+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:47.642 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:47.641+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:48.533 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:48.531+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:48.602 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:48.601+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:49.562 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:49.561+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:49.580 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:49.579+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:50.578 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:50.576+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:50.587 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:50.586+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:51.551 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:51.549+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:51.571 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:51.570+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:52.557 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:52.555+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:52.563 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:52.561+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:53.531 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:53.529+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:53.557 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:53.555+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:54.482 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:54.480+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:54.530 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:54.529+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:55.498 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:55.496+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:55.505 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:55.503+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:56.471 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:56.469+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:56.508 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:56.507+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:57.499 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:57.497+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:57.507 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:57.505+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:58.474 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:58.472+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:58.497 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:58.495+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:59.438 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:45:59.436+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:45:59.520 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:45:59.518+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:00.414 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:00.413+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:00.495 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:00.494+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:01.379 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:01.378+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:01.523 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:01.522+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:02.406 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:02.405+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:02.477 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:02.476+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:03.405 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:03.404+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:03.456 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:03.456+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:04.439 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:04.438+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:04.469 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:04.468+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:05.421 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:05.419+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:05.477 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:05.476+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:06.412 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:06.411+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:06.515 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:06.515+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:07.383 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:07.382+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:07.486 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:07.485+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:08.375 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:08.374+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:08.444 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:08.443+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:09.424 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:09.423+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:09.471 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:09.470+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:10.451 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:10.450+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:10.461 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:10.460+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:11.416 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:11.416+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:11.473 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:11.472+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:12.389 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:12.388+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:12.513 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:12.512+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:13.422 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:13.422+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:13.490 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:13.490+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:14.454 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:14.453+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:14.509 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:14.509+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:15.463 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:15.462+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:15.543 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:15.543+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:16.429 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:16.428+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:16.540 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:16.540+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:17.436 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:17.436+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:17.518 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:17.518+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:18.471 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:18.470+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:18.529 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:18.528+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:19.468 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:19.468+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:19.552 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:19.551+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:20.454 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:20.453+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:20.539 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:20.537+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:21.418 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:21.417+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:21.536 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:21.534+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:22.434 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:22.432+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:22.532 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:22.531+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:23.408 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:23.406+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:23.566 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:23.564+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:24.418 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:24.416+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:24.565 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:24.564+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:25.404 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:25.402+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:25.524 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:25.523+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:26.425 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:26.424+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:26.492 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:26.491+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:27.475 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:27.473+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:27.506 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:27.504+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:28.501 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:28.500+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:28.552 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:28.551+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:29.470 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:29.468+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:29.517 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:29.515+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:30.429 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:30.428+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:30.559 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:30.557+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:31.447 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:31.445+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:31.583 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:31.582+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:32.484 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:32.482+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:32.547 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:32.545+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:33.509 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:33.507+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:33.550 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:33.548+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:34.540 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:34.538+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:34.546 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:34.545+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:35.589 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:35.587+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:35.590 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:35.589+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:36.620 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:36.618+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:36.637 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:36.635+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:37.643 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:37.641+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:37.684 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:37.682+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:38.664 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:38.663+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:38.678 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:38.677+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:39.636 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:39.634+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:39.645 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:39.644+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:40.595 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:40.593+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:40.682 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:40.681+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:41.591 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:41.589+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:41.725 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:41.724+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:42.549 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:42.547+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:42.716 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:42.715+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:43.547 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:43.545+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:43.709 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:43.708+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:44.574 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:44.572+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:44.683 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:44.681+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:45.616 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:45.614+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:45.720 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:45.718+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:46.625 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:46.623+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:46.731 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:46.729+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:47.587 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:47.585+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:47.760 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:47.758+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:48.616 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:48.614+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:48.755 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:48.754+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:49.631 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:49.629+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:49.797 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:49.796+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:50.625 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:50.623+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:50.763 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:50.761+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:51.650 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:51.648+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:51.809 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:51.808+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:52.678 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:52.676+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:52.848 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:52.847+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:53.660 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:53.659+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:53.840 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:53.838+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:54.681 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:54.679+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:54.811 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:54.809+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:55.669 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:55.667+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:55.842 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:55.840+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:56.719 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:56.717+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:56.882 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:56.880+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:57.744 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:57.743+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:57.840 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:57.838+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:58.707 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:58.706+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:58.791 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:58.790+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:59.660 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:46:59.659+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:46:59.841 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:46:59.841+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:00.648 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:00.647+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:00.798 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:00.797+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:01.624 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:01.623+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:01.803 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:01.803+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:02.633 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:02.632+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:02.804 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:02.803+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:03.658 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:03.657+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:03.793 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:03.792+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:04.651 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:04.651+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:04.781 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:04.781+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:05.654 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:05.653+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:05.800 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:05.799+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:06.691 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:06.690+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:06.755 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:06.754+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:07.702 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:07.701+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:07.771 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:07.771+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:08.717 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:08.717+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:08.752 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:08.752+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:09.742 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:09.741+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:09.783 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:09.782+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:10.709 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:10.708+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:10.801 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:10.800+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:11.674 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:11.673+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:11.761 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:11.760+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:12.702 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:12.701+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:12.714 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:12.714+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:13.688 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:13.687+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:13.704 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:13.703+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:14.673 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:14.672+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:14.675 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:14.674+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:15.674 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:15.673+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:15.679 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:15.679+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:16.664 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:16.663+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:16.694 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:16.693+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:17.620 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:17.619+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:17.685 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:17.684+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:18.651 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:18.649+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:18.688 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:18.686+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:19.613 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:19.611+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:19.709 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:19.708+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:20.571 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:20.570+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:20.698 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:20.696+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:21.544 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:21.542+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:21.672 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:21.671+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:22.563 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:22.561+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:22.671 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:22.670+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:23.570 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:23.569+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:23.705 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:23.703+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:24.569 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:24.567+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:24.709 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:24.707+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:25.608 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:25.606+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:25.733 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:25.731+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:26.564 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:26.562+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:26.697 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:26.695+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:27.524 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:27.522+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:27.661 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:27.659+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:28.574 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:28.570+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:28.658 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:28.656+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:29.564 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:29.563+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:29.622 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:29.620+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:30.526 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:30.524+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:30.627 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:30.626+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:31.574 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:31.572+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:31.581 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:31.579+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:32.588 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:32.586+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:32.619 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:32.618+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:33.601 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:33.599+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:33.609 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:33.608+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:34.568 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:34.566+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:34.591 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:34.590+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:35.565 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:35.563+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:35.628 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:35.626+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:36.546 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:36.544+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:36.590 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:36.588+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:37.530 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:37.528+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:37.601 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:37.599+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:38.575 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:38.574+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:38.634 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:38.632+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:39.535 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:39.533+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:39.655 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:39.653+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:40.538 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:40.536+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:40.629 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:40.628+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:41.553 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:41.552+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:41.630 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:41.629+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:42.539 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:42.538+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:42.638 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:42.636+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:43.516 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:43.514+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:43.655 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:43.653+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:44.513 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:44.512+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:44.629 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:44.627+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:45.547 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:45.545+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:45.650 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:45.648+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:46.528 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:46.526+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:46.612 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:46.610+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:47.505 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:47.503+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:47.571 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:47.569+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:48.488 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:48.486+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:48.619 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:48.617+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:49.510 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:49.508+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:49.627 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:49.626+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:50.483 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:50.482+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:50.611 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:50.610+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:51.455 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:51.453+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:51.614 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:51.612+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:52.445 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:52.444+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:52.585 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:52.584+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:53.460 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:53.458+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:53.542 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:53.541+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:54.436 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:54.434+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:54.593 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:54.591+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:55.464 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:55.462+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:55.583 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:55.583+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:56.441 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:56.439+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:56.542 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:56.541+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:57.393 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:57.392+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:57.544 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:57.543+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:58.365 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:58.364+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:58.527 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:58.526+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:59.382 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:47:59.381+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:47:59.575 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:47:59.575+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:00.352 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:00.351+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:00.546 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:00.546+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:01.400 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:01.400+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:01.538 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:01.535+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:02.440 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:02.439+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:02.538 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:02.537+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:03.429 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:03.428+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:03.568 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:03.567+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:04.431 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:04.430+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:04.579 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:04.578+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:05.383 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:05.382+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:05.574 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:05.574+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:06.350 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:06.350+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:06.582 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:06.581+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:07.305 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:07.305+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:07.584 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:07.584+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:08.274 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:08.274+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:08.580 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:08.579+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:09.279 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:09.278+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:09.541 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:09.540+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:10.287 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:10.286+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:10.513 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:10.513+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:11.248 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:11.248+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:11.518 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:11.518+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:12.249 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:12.248+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:12.509 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:12.508+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:13.298 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:13.298+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:13.494 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:13.493+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:14.341 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:14.340+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:14.450 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:14.450+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:15.298 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:15.298+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:15.462 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:15.462+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:16.332 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:16.332+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:16.463 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:16.463+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:17.368 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:17.366+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:17.512 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:17.510+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:18.389 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:18.387+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:18.532 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:18.531+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:19.385 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:19.384+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:19.530 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:19.528+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:20.364 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:20.362+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:20.522 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:20.521+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:21.350 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:21.349+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:21.556 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:21.554+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:22.326 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:22.324+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:22.539 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:22.537+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:23.333 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:23.331+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:23.527 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:23.525+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:24.361 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:24.359+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:24.520 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:24.519+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:25.404 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:25.403+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:25.541 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:25.539+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:26.356 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:26.355+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:26.576 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:26.574+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:27.352 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:27.351+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:27.575 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:27.574+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:28.357 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:28.355+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:28.537 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:28.536+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:29.339 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:29.338+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:29.566 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:29.564+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:30.310 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:30.309+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:30.525 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:30.522+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:31.279 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:31.277+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:31.570 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:31.569+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:32.262 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:32.260+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:32.534 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:32.532+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:33.221 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:33.219+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:33.492 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:33.490+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:34.180 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:34.178+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:34.479 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:34.478+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:35.145 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:35.143+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:35.462 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:35.461+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:36.190 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:36.188+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:36.456 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:36.455+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:37.177 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:37.175+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:37.490 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:37.488+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:38.199 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:38.197+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:38.483 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:38.482+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:39.222 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:39.220+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:39.500 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:39.498+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:40.247 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:40.245+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:40.460 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:40.458+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:41.219 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:41.217+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:41.481 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:41.479+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:42.224 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:42.222+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:42.487 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:42.463+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:43.190 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:43.188+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:43.500 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:43.499+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:44.217 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:44.215+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:44.484 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:44.483+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:45.179 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:45.177+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:45.533 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:45.531+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:46.218 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:46.216+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:46.575 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:46.574+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:47.260 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:47.258+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:47.574 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:47.572+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:48.231 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:48.229+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:48.541 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:48.539+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:49.276 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:49.274+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:49.584 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:49.582+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:50.229 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:50.227+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:50.541 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:50.539+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:51.246 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:51.244+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:51.591 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:51.589+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:52.271 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:52.269+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:52.589 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:52.588+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:53.310 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:53.308+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:53.637 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:53.635+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:54.338 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:54.337+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:54.643 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:54.641+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:55.325 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:55.324+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:55.663 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:55.662+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:56.328 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:56.327+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:56.660 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:56.659+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:57.336 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:57.335+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:57.697 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:57.696+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:58.299 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:58.299+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:58.709 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:58.708+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:59.279 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:48:59.278+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:48:59.753 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:48:59.752+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:00.251 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:00.251+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:00.747 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:00.746+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:01.260 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:01.260+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:01.754 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:01.753+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:02.277 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:02.277+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:02.720 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:02.720+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:03.315 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:03.315+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:03.718 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:03.717+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:04.312 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:04.312+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:04.705 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:04.705+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:05.361 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:05.360+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:05.756 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:05.755+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:06.362 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:06.362+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:06.774 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:06.773+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:07.319 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:07.318+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:07.816 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:07.816+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:08.351 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:08.351+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:08.786 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:08.786+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:09.384 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:09.384+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:09.776 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:09.775+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:10.405 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:10.404+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:10.749 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:10.748+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:11.437 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:11.436+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:11.718 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:11.718+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:12.453 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:12.453+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:12.684 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:12.683+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:13.439 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:13.438+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:13.672 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:13.671+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:14.400 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:14.399+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:14.641 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:14.641+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:15.374 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:15.372+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:15.594 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:15.593+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:16.398 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:16.397+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:16.596 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:16.595+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:17.443 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:17.441+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:17.618 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:17.617+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:18.421 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:18.419+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:18.604 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:18.602+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:19.417 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:19.415+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:19.579 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:19.578+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:20.443 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:20.441+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:20.539 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:20.537+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:21.405 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:21.403+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:21.524 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:21.522+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:22.377 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:22.375+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:22.565 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:22.563+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:23.396 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:23.394+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:23.527 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:23.526+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:24.398 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:24.396+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:24.494 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:24.493+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:25.353 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:25.352+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:25.460 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:25.459+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:26.340 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:26.338+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:26.435 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:26.434+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:27.367 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:27.365+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:27.485 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:27.483+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:28.378 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:28.377+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:28.526 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:28.524+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:29.349 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:29.347+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:29.480 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:29.479+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:30.392 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:30.390+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:30.441 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:30.439+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:31.419 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:31.417+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:31.433 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:31.432+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:32.392 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:32.390+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:32.479 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:32.478+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:33.443 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:33.441+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:33.524 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:33.522+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:34.419 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:34.417+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:34.531 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:34.530+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:35.439 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:35.437+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:35.527 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:35.525+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:36.454 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:36.453+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:36.528 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:36.526+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:37.464 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:37.462+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:37.551 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:37.550+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:38.511 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:38.510+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:38.547 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:38.545+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:39.546 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:39.545+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:39.552 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:39.550+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:40.503 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:40.501+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:40.587 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:40.585+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:41.458 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:41.456+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:41.561 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:41.559+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:42.492 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:42.490+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:42.575 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:42.574+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:43.516 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:43.514+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:43.588 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:43.587+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:44.517 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:44.515+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:44.612 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:44.610+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:45.559 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:45.557+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:45.563 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:45.561+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:46.545 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:46.543+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:46.586 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:46.585+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:47.507 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:47.505+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:47.583 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:47.581+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:48.534 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:48.532+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:48.619 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:48.618+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:49.561 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:49.560+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:49.668 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:49.666+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:50.528 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:50.526+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:50.677 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:50.675+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:51.479 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:51.477+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:51.696 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:51.694+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:52.475 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:52.473+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:52.701 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:52.699+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:53.522 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:53.522+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:53.717 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:53.716+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:54.515 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:54.515+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:54.703 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:54.702+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:55.516 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:55.515+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:55.734 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:55.733+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:56.553 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:56.553+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:56.758 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:56.758+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:57.524 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:57.523+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:57.715 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:57.715+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:58.481 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:58.480+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:58.700 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:58.700+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:59.461 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:49:59.460+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:49:59.685 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:49:59.685+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:00.491 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:00.490+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:00.665 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:00.665+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:01.462 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:01.462+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:01.649 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:01.649+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:02.495 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:02.494+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:02.675 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:02.675+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:03.470 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:03.469+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:03.671 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:03.670+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:04.503 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:04.503+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:04.638 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:04.637+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:05.501 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:05.501+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:05.656 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:05.655+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:06.527 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:06.526+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:06.660 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:06.659+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:07.511 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:07.511+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:07.615 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:07.614+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:08.559 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:08.558+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:08.595 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:08.594+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:09.572 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:09.571+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:09.737 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:09.583+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:10.553 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:10.552+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:10.587 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:10.586+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:11.556 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:11.555+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:11.568 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:11.568+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:12.566 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:12.565+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:12.603 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:12.603+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:13.532 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:13.530+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:13.624 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:13.622+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:14.568 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:14.564+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:14.581 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:14.579+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:15.551 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:15.550+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:15.612 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:15.610+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:16.580 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:16.578+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:16.595 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:16.594+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:17.547 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:17.546+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:17.585 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:17.584+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:18.575 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:18.574+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:18.588 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:18.586+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:19.589 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:19.587+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:19.611 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:19.609+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:20.596 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:20.594+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:20.637 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:20.636+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:21.547 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:21.546+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:21.626 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:21.624+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:22.498 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:22.497+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:22.669 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:22.668+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:23.483 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:23.481+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:23.672 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:23.670+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:24.517 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:24.515+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:24.691 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:24.689+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:25.534 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:25.533+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:25.667 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:25.665+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:26.523 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:26.522+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:26.683 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:26.681+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:27.512 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:27.510+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:27.726 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:27.724+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:28.561 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:28.559+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:28.721 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:28.719+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:29.596 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:29.594+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:29.762 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:29.760+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:30.627 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:30.626+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:30.780 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:30.779+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:31.631 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:31.630+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:31.767 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:31.765+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:32.664 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:32.662+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:32.764 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:32.763+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:33.709 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:33.707+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:33.786 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:33.784+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:34.690 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:34.689+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:34.764 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:34.762+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:35.675 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:35.674+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:35.794 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:35.792+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:36.701 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:36.699+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:36.752 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:36.751+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:37.655 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:37.653+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:37.797 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:37.796+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:38.645 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:38.644+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:38.798 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:38.797+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:39.611 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:39.610+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:39.841 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:39.839+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:40.643 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:40.641+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:40.856 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:40.854+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:41.657 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:41.655+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:41.813 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:41.812+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:42.704 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:42.702+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:42.827 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:42.825+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:43.658 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:43.656+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:43.877 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:43.875+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:44.642 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:44.640+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:44.925 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:44.923+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:45.666 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:45.664+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:45.962 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:45.960+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:46.628 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:46.626+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:46.991 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:46.989+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:47.623 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:47.622+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:47.971 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:47.969+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:48.629 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:48.627+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:48.958 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:48.956+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:49.623 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:49.622+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:49.986 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:49.985+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:50.665 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:50.664+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:50.941 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:50.939+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:51.680 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:51.680+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:51.951 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:51.950+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:52.703 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:52.702+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:52.971 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:52.970+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:53.693 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:53.692+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:53.965 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:53.964+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:54.664 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:54.664+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:54.986 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:54.986+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:55.616 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:55.616+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:55.950 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:55.950+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:56.575 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:56.574+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:56.938 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:56.937+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:57.617 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:57.616+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:57.930 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:57.930+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:58.577 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:58.577+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:58.962 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:58.961+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:59.596 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:50:59.596+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:50:59.934 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:50:59.933+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:00.615 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:00.614+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:00.975 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:00.974+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:01.629 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:01.628+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:01.963 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:01.962+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:02.593 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:02.592+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:02.966 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:02.965+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:03.584 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:03.584+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:03.934 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:03.934+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:04.583 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:04.582+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:04.935 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:04.934+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:05.614 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:05.614+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:05.901 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:05.900+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:06.614 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:06.613+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:06.935 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:06.934+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:07.593 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:07.592+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:07.928 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:07.928+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:08.590 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:08.589+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:08.900 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:08.900+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:09.548 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:09.546+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:09.940 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:09.940+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:10.562 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:10.561+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:10.954 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:10.954+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:11.568 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:11.567+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:11.950 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:11.949+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:12.525 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:12.523+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:12.941 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:12.940+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:13.501 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:13.499+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:13.979 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:13.977+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:14.498 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:14.496+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:14.998 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:14.996+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:15.519 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:15.517+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:15.993 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:15.991+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:16.558 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:16.557+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:16.967 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:16.965+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:17.525 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:17.524+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:17.966 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:17.964+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:18.562 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:18.560+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:18.933 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:18.932+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:19.603 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:19.601+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:19.965 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:19.963+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:20.604 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:20.603+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:20.987 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:20.985+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:21.632 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:21.631+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:21.973 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:21.971+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:22.673 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:22.671+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:23.001 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:22.999+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:23.664 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:23.663+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:24.028 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:24.026+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:24.622 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:24.620+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:25.057 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:25.056+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:25.589 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:25.588+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:26.076 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:26.075+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:26.544 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:26.542+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:27.027 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:27.025+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:27.573 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:27.571+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:28.075 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:28.073+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:28.608 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:28.606+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:29.071 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:29.069+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:29.645 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:29.643+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:30.068 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:30.066+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:30.662 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:30.660+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:31.099 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:31.097+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:31.642 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:31.640+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:32.091 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:32.089+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:32.693 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:32.690+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:33.122 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:33.121+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:33.693 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:33.689+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:34.122 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:34.121+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:34.691 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:34.689+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:35.151 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:35.149+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:35.663 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:35.662+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:36.130 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:36.128+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:36.710 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:36.708+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:37.099 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:37.097+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:37.726 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:37.724+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:38.136 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:38.135+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:38.706 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:38.704+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:39.159 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:39.158+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:39.693 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:39.691+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:40.197 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:40.195+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:40.662 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:40.661+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:41.159 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:41.157+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:41.688 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:41.686+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:42.203 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:42.201+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:42.716 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:42.715+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:43.217 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:43.215+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:43.711 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:43.710+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:44.250 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:44.249+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:44.743 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:44.741+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:45.270 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:45.269+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:45.786 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:45.782+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:46.281 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:46.280+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:46.762 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:46.761+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:47.281 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:47.279+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:47.751 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:47.750+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:48.235 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:48.233+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:48.792 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:48.790+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:49.199 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:49.198+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:49.778 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:49.776+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:50.237 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:50.237+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:50.745 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:50.744+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:51.218 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:51.217+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:51.743 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:51.743+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:52.195 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:52.194+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:52.710 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:52.709+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:53.225 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:53.224+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:53.698 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:53.697+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:54.228 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:54.227+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:54.705 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:54.704+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:55.265 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:55.264+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:55.740 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:55.739+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:56.299 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:56.298+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:56.758 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:56.757+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:57.350 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:57.349+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:57.776 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:57.776+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:58.362 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:58.361+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:58.728 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:58.728+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:59.402 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:51:59.402+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:51:59.711 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:51:59.711+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:00.359 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:00.358+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:00.675 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:00.675+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:01.313 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:01.312+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:01.672 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:01.671+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:02.294 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:02.294+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:02.717 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:02.716+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:03.286 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:03.285+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:04.160 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:03.683+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:04.328 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:04.327+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:04.779 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:04.687+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:05.290 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:05.289+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:05.648 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:05.647+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:06.247 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:06.246+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:06.685 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:06.685+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:07.286 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:07.285+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:07.715 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:07.714+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:08.303 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:08.303+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:08.693 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:08.693+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:09.302 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:09.302+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:09.684 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:09.684+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:10.310 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:10.308+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:10.681 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:10.679+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:11.262 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:11.260+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:11.719 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:11.718+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:12.258 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:12.256+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:12.770 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:12.768+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:13.222 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:13.220+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:13.763 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:13.761+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:14.244 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:14.242+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:14.801 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:14.799+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:15.286 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:15.284+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:15.755 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:15.754+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:16.298 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:16.296+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:16.715 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:16.714+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:17.321 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:17.320+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:17.693 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:17.691+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:18.365 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:18.363+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:18.731 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:18.730+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:19.384 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:19.382+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:19.756 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:19.754+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:20.393 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:20.392+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:20.794 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:20.793+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:21.400 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:21.398+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:21.791 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:21.789+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:22.433 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:22.432+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:22.750 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:22.748+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:23.457 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:23.456+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:23.758 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:23.756+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:24.505 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:24.503+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:24.787 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:24.786+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:25.475 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:25.473+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:25.781 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:25.779+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:26.471 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:26.469+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:26.745 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:26.743+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:27.423 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:27.421+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:27.747 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:27.745+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:28.384 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:28.382+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:28.776 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:28.774+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:29.358 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:29.356+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:29.758 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:29.756+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:30.398 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:30.396+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:30.788 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:30.787+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:31.444 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:31.442+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:31.763 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:31.762+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:32.422 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:32.421+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:32.753 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:32.752+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:33.459 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:33.457+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:33.735 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:33.733+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:34.483 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:34.481+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:34.692 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:34.690+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:35.444 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:35.442+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:35.710 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:35.708+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:36.493 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:36.491+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:36.704 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:36.702+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:37.495 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:37.494+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:37.696 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:37.695+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:38.451 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:38.450+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:38.656 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:38.654+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:39.450 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:39.448+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:39.610 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:39.608+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:40.464 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:40.463+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:40.591 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:40.589+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:41.429 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:41.427+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:41.617 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:41.616+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:42.471 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:42.469+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:42.593 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:42.591+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:43.489 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:43.488+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:43.600 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:43.598+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:44.457 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:44.455+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:44.572 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:44.571+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:45.439 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:45.438+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:45.613 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:45.612+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:46.454 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:46.452+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:46.595 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:46.593+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:47.465 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:47.463+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:47.597 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:47.596+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:48.478 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:48.478+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:48.559 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:48.558+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:49.484 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:49.484+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:49.550 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:49.550+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:50.505 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:50.504+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:50.517 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:50.516+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:51.486 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:51.486+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:51.494 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:51.494+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:52.480 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:52.479+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:52.486 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:52.486+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:53.448 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:53.447+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:53.477 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:53.476+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:54.467 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:54.467+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:54.492 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:54.492+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:55.474 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:55.474+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:55.521 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:55.520+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:56.493 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:56.492+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:56.498 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:56.497+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:57.453 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:57.452+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:57.484 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:57.483+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:58.486 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:58.486+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:58.497 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:58.496+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:59.485 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:52:59.484+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:52:59.530 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:52:59.530+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:00.526 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:00.524+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:00.557 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:00.557+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:01.572 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:01.571+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:01.604 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:01.604+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:02.597 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:02.596+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:03.087 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:02.615+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:03.552 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:03.552+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:03.589 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:03.588+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:04.518 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:04.517+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:04.605 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:04.604+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:05.532 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:05.532+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:05.617 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:05.616+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:06.534 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:06.533+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:06.583 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:06.582+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:07.557 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:07.557+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:07.600 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:07.600+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:08.555 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:08.554+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:08.608 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:08.608+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:09.560 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:09.558+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:09.580 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:09.578+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:10.550 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:10.548+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:10.576 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:10.575+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:11.590 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:11.589+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:11.600 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:11.599+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:12.578 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:12.576+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:12.621 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:12.620+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:13.596 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:13.594+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:13.596 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:13.594+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:14.570 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:14.568+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:14.627 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:14.625+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:15.578 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:15.576+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:15.628 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:15.627+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:16.592 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:16.591+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:16.630 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:16.629+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:17.553 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:17.552+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:17.644 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:17.643+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:18.565 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:18.563+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:18.620 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:18.618+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:19.557 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:19.556+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:19.592 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:19.590+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:20.570 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:20.569+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:20.588 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:20.587+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:21.552 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:21.551+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:21.628 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:21.627+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:22.508 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:22.506+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:22.628 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:22.627+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:23.548 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:23.546+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:23.666 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:23.664+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:24.576 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:24.575+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:24.706 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:24.704+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:25.560 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:25.559+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:25.671 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:25.669+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:26.534 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:26.532+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:26.661 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:26.659+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:27.571 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:27.570+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:27.683 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:27.681+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:28.539 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:28.537+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:28.731 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:28.729+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:29.535 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:29.533+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:29.701 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:29.699+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:30.514 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:30.513+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:30.668 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:30.666+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:31.518 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:31.516+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:31.631 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:31.629+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:32.522 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:32.520+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:32.632 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:32.630+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:33.499 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:33.498+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:33.632 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:33.631+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:34.503 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:34.502+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:34.615 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:34.613+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:35.537 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:35.535+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:35.626 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:35.624+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:36.540 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:36.538+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:36.614 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:36.613+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:37.550 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:37.548+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:37.641 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:37.640+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:38.543 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:38.541+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:38.622 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:38.620+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:39.550 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:39.548+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:39.665 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:39.663+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:40.547 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:40.546+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:40.686 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:40.684+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:41.507 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:41.505+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:41.718 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:41.716+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:42.502 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:42.501+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:42.747 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:42.746+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:43.480 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:43.478+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:43.798 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:43.797+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:44.436 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:44.435+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:44.768 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:44.766+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:45.408 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:45.406+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:45.800 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:45.799+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:46.422 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:46.422+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:46.808 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:46.808+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:47.374 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:47.373+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:47.779 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:47.778+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:48.392 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:48.391+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:48.764 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:48.763+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:49.356 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:49.356+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:49.718 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:49.718+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:50.310 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:50.309+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:50.681 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:50.681+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:51.305 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:51.304+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:51.690 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:51.689+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:52.269 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:52.269+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:52.696 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:52.696+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:53.319 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:53.318+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:53.671 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:53.670+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:54.309 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:54.308+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:54.696 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:54.695+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:55.313 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:55.311+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:55.668 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:55.668+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:56.276 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:56.275+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:56.638 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:56.638+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:57.268 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:57.268+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:57.675 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:57.674+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:58.230 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:58.229+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:58.724 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:58.723+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:59.242 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:53:59.241+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:53:59.689 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:53:59.688+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:00.257 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:00.257+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:00.715 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:00.714+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:01.224 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:01.223+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:01.694 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:01.693+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:02.269 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:02.268+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:02.741 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:02.740+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:03.228 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:03.227+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:03.710 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:03.710+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:04.194 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:04.194+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:04.706 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:04.706+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:05.165 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:05.164+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:05.673 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:05.673+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:06.158 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:06.158+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:06.707 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:06.706+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:07.138 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:07.137+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:07.713 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:07.711+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:08.096 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:08.094+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:08.762 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:08.760+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:09.131 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:09.129+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:09.721 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:09.719+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:10.164 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:10.163+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:10.683 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:10.682+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:11.176 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:11.175+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:11.678 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:11.677+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:12.191 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:12.189+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:12.681 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:12.679+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:13.232 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:13.231+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:13.728 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:13.727+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:14.237 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:14.235+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:14.698 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:14.697+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:15.277 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:15.275+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:15.680 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:15.677+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:16.231 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:16.229+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:16.708 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:16.706+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:17.231 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:17.229+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:17.680 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:17.679+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:18.232 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:18.230+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:18.711 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:18.709+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:19.274 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:19.273+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:19.745 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:19.744+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:20.247 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:20.245+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:20.709 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:20.707+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:21.198 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:21.197+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:21.674 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:21.672+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:22.225 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:22.224+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:22.690 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:22.689+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:23.197 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:23.195+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:23.682 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:23.680+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:24.206 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:24.205+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:24.641 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:24.639+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:25.207 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:25.205+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:25.664 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:25.663+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:26.227 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:26.225+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:26.658 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:26.657+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:27.276 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:27.275+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:27.643 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:27.641+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:28.296 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:28.294+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:28.661 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:28.659+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:29.274 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:29.272+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:29.707 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:29.706+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:30.246 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:30.245+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:30.712 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:30.710+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:31.268 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:31.267+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:31.736 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:31.735+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:32.280 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:32.278+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:32.776 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:32.774+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:33.323 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:33.321+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:33.734 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:33.732+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:34.328 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:34.326+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:34.784 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:34.783+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:35.284 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:35.282+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:35.737 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:35.735+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:36.331 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:36.329+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:36.696 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:36.694+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:37.298 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:37.297+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:37.697 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:37.695+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:38.329 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:38.327+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:38.713 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:38.711+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:39.290 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:39.288+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:39.739 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:39.738+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:40.309 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:40.307+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:40.757 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:40.755+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:41.304 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:41.303+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:41.794 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:41.792+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:42.324 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:42.322+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:42.819 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:42.817+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:43.365 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:43.363+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:43.804 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:43.803+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:44.329 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:44.328+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:44.765 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:44.764+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:45.293 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:45.292+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:45.748 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:45.748+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:46.253 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:46.253+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:46.722 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:46.721+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:47.216 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:47.216+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:47.771 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:47.770+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:48.249 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:48.248+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:48.761 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:48.761+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:49.209 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:49.209+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:49.722 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:49.721+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:50.189 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:50.188+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:50.723 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:50.722+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:51.164 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:51.163+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:51.699 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:51.698+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:52.180 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:52.179+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:52.716 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:52.715+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:53.206 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:53.205+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:53.703 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:53.702+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:54.162 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:54.161+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:54.747 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:54.746+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:55.156 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:55.156+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:55.758 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:55.757+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:56.131 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:56.130+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:56.710 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:56.709+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:57.155 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:57.154+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:57.682 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:57.681+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:58.154 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:58.153+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:58.717 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:58.717+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:59.180 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:54:59.179+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:54:59.719 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:54:59.719+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:00.188 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:00.187+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:00.739 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:00.738+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:01.218 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:01.217+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:01.715 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:01.715+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:02.197 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:02.196+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:02.685 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:02.685+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:03.199 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:03.198+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:03.679 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:03.679+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:04.221 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:04.220+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:04.713 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:04.713+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:05.262 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:05.261+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:05.677 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:05.675+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:06.261 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:06.259+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:06.646 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:06.644+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:07.302 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:07.301+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:07.652 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:07.650+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:08.337 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:08.336+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:08.699 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:08.698+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:09.360 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:09.359+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:09.743 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:09.742+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:10.337 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:10.336+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:10.782 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:10.780+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:11.354 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:11.352+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:11.769 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:11.768+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:12.382 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:12.380+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:12.769 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:12.768+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:13.371 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:13.369+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:13.776 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:13.774+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:14.383 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:14.382+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:14.774 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:14.772+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:15.335 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:15.334+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:15.795 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:15.794+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:16.294 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:16.292+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:16.846 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:16.844+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:17.265 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:17.264+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:17.882 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:17.881+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:18.250 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:18.249+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:18.914 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:18.912+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:19.285 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:19.283+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:19.956 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:19.954+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:20.305 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:20.304+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:20.925 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:20.924+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:21.276 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:21.274+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:21.958 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:21.956+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:22.259 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:22.258+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:22.955 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:22.953+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:23.302 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:23.300+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:23.929 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:23.928+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:24.307 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:24.306+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:24.979 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:24.977+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:25.328 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:25.326+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:26.005 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:26.003+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:26.317 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:26.316+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:26.983 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:26.981+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:27.353 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:27.351+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:27.946 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:27.944+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:28.333 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:28.332+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:28.993 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:28.992+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:29.329 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:29.327+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:30.037 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:30.035+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:30.351 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:30.349+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:31.029 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:31.027+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:31.303 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:31.302+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:32.068 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:32.067+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:32.300 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:32.298+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:33.042 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:33.040+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:33.324 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:33.323+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:34.025 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:34.023+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:34.308 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:34.307+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:35.061 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:35.059+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:35.274 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:35.273+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:36.014 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:36.012+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:36.282 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:36.280+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:37.051 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:37.049+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:37.318 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:37.317+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:38.035 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:38.033+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:38.294 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:38.292+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:39.075 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:39.073+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:39.288 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:39.287+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:40.073 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:40.072+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:40.333 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:40.332+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:41.026 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:41.024+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:41.323 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:41.321+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:42.058 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:42.056+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:42.293 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:42.291+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:43.084 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:43.082+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:43.281 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:43.280+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:44.057 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:44.056+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:44.267 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:44.266+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:45.050 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:45.049+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:45.281 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:45.280+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:46.074 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:46.073+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:46.289 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:46.288+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:47.042 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:47.042+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:47.287 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:47.287+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:48.084 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:48.084+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:48.336 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:48.335+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:49.058 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:49.057+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:49.377 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:49.376+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:50.070 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:50.069+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:50.381 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:50.381+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:51.103 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:51.103+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:51.408 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:51.407+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:52.123 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:52.123+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:52.410 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:52.410+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:53.128 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:53.128+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:53.461 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:53.460+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:54.154 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:54.154+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:54.492 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:54.491+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:55.189 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:55.188+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:55.518 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:55.518+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:56.226 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:56.225+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:56.505 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:56.504+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:57.192 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:57.192+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:57.459 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:57.458+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:58.217 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:58.216+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:58.499 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:58.498+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:59.250 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:55:59.250+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:55:59.512 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:55:59.511+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:00.238 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:00.237+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:00.560 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:00.559+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:01.216 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:01.215+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:01.595 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:01.595+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:02.292 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:02.175+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:02.605 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:02.604+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:03.177 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:03.176+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:03.570 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:03.569+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:04.203 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:04.202+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:04.552 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:04.551+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:05.159 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:05.158+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:05.520 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:05.519+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:06.200 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:06.199+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:06.487 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:06.485+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:07.171 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:07.169+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:07.470 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:07.469+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:08.132 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:08.131+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:08.427 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:08.426+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:09.132 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:09.131+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:09.416 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:09.414+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:10.100 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:10.098+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:10.391 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:10.389+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:11.053 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:11.052+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:11.350 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:11.348+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:12.030 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:12.028+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:12.329 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:12.328+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:12.983 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:12.981+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:13.325 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:13.323+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:13.953 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:13.951+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:14.317 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:14.315+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:14.944 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:14.943+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:15.361 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:15.360+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:15.909 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:15.907+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:16.379 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:16.378+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:16.885 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:16.883+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:17.372 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:17.370+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:17.856 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:17.855+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:18.366 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:18.364+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:18.886 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:18.877+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:19.414 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:19.412+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:19.852 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:19.851+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:20.449 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:20.448+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:20.809 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:20.807+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:21.443 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:21.441+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:21.820 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:21.818+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:22.418 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:22.417+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:22.814 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:22.811+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:23.426 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:23.425+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:23.777 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:23.775+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:24.391 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:24.390+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:24.801 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:24.800+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:25.415 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:25.413+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:25.802 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:25.800+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:26.379 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:26.377+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:26.806 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:26.804+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:27.395 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:27.393+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:27.808 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:27.806+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:28.346 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:28.345+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:28.858 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:28.856+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:28.940 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-05T23:56:28.939+0000 7f994396e700 0 -- 172.21.15.167:0/1156974075 >> [v2:172.21.15.167:6816/144114,v1:172.21.15.167:6817/144114] conn(0x7f9928034140 msgr2=0x7f9928036520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2023-10-05T23:56:29.361 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:29.359+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:29.840 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:29.839+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:30.323 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:30.322+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:30.861 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:30.860+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:31.300 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:31.299+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:31.844 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:31.843+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:32.311 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:32.309+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:32.844 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:32.843+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:33.354 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:33.353+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:33.819 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:33.817+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:34.391 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:34.389+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:34.784 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:34.782+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:35.379 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:35.377+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:35.816 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:35.814+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:36.387 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:36.385+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:36.786 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:36.785+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:37.388 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:37.386+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:37.833 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:37.831+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:38.364 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:38.362+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:38.797 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:38.795+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:39.330 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:39.328+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:39.815 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:39.814+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:40.335 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:40.334+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:40.845 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:40.843+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:41.311 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:41.310+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:41.820 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:41.820+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:42.276 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:42.275+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:42.793 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:42.793+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:43.247 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:43.247+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:43.823 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:43.823+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:44.272 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:44.271+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:44.833 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:44.832+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:45.249 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:45.248+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:45.849 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:45.849+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:46.220 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:46.219+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:46.838 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:46.837+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:47.230 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:47.230+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:47.879 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:47.878+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:48.246 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:48.246+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:48.842 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:48.842+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:49.277 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:49.277+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:49.892 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:49.892+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:50.306 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:50.306+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:50.865 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:50.864+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:51.343 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:51.342+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:51.823 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:51.823+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:52.301 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:52.300+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:52.779 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:52.779+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:53.281 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:53.281+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:53.826 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:53.826+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:54.242 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:54.241+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:54.800 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:54.799+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:55.206 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:55.206+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:55.808 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:55.808+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:56.212 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:56.211+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:56.816 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:56.816+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:57.178 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:57.177+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:57.769 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:57.769+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:58.215 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:58.214+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:58.729 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:58.728+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:59.206 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:56:59.205+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:56:59.731 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:56:59.731+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:00.231 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:00.231+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:00.745 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:00.744+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:01.266 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:01.265+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:01.747 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:01.747+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:02.219 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:02.217+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:02.747 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:02.745+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:03.242 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:03.240+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:03.701 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:03.699+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:04.263 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:04.262+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:04.685 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:04.683+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:05.263 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:05.262+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:05.670 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:05.669+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:06.303 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:06.302+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:06.656 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:06.654+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:07.352 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:07.351+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:07.659 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:07.658+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:08.345 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:08.343+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:08.703 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:08.702+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:09.324 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:09.323+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:09.701 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:09.699+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:10.310 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:10.308+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:10.689 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:10.688+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:11.324 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:11.323+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:11.683 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:11.681+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:12.357 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:12.355+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:12.724 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:12.723+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:13.376 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:13.374+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:13.719 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:13.717+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:14.366 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:14.364+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:14.752 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:14.750+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:15.363 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:15.362+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:15.766 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:15.764+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:16.358 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:16.356+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:16.729 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:16.727+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:17.327 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:17.326+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:17.685 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:17.683+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:18.375 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:18.374+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:18.721 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:18.719+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:19.340 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:19.338+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:19.753 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:19.752+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:20.307 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:20.305+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:20.738 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:20.733+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:21.314 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:21.313+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:21.747 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:21.746+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:22.318 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:22.316+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:22.722 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:22.721+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:23.285 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:23.284+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:23.726 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:23.724+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:24.305 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:24.303+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:24.699 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:24.698+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:25.274 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:25.272+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:25.680 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:25.678+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:26.274 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:26.272+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:26.715 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:26.713+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:27.319 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:27.318+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:27.699 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:27.697+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:28.292 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:28.290+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:28.653 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:28.651+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:29.309 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:29.307+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:29.657 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:29.656+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:30.329 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:30.327+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:30.644 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:30.642+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:31.342 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:31.340+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:31.653 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:31.651+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:32.386 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:32.385+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:32.613 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:32.611+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:33.358 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:33.357+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:33.661 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:33.659+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:34.309 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:34.308+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:34.687 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:34.686+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:35.317 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:35.315+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:35.671 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:35.670+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:36.309 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:36.307+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:36.662 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:36.660+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:37.289 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:37.288+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:37.619 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:37.618+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:38.300 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:38.298+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:38.660 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:38.659+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:39.333 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:39.332+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:39.653 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:39.652+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:40.295 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:40.294+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:40.623 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:40.623+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:41.260 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:41.259+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:41.654 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:41.653+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:42.223 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:42.223+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:42.697 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:42.696+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:43.219 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:43.218+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:43.672 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:43.671+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:44.185 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:44.184+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:44.654 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:44.653+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:45.177 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:45.176+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:45.612 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:45.611+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:46.153 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:46.152+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:46.598 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:46.598+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:47.198 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:47.197+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:47.598 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:47.597+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:48.215 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:48.214+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:48.624 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:48.623+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:49.212 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:49.211+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:49.575 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:49.574+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:50.248 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:50.248+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:50.545 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:50.544+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:51.220 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:51.219+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:51.502 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:51.501+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:52.267 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:52.267+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:52.484 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:52.483+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:53.229 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:53.229+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:53.481 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:53.480+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:54.267 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:54.267+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:54.442 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:54.441+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:55.292 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:55.292+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:55.425 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:55.424+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:56.272 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:56.271+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:56.417 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:56.417+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:57.272 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:57.271+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:57.452 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:57.451+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:58.266 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:58.265+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:58.403 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:58.402+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:59.239 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:57:59.238+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:57:59.427 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:57:59.427+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:00.207 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:00.207+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:00.465 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:00.464+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:01.256 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:01.254+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:01.431 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:01.429+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:02.227 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:02.225+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:02.471 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:02.470+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:03.233 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:03.231+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:03.483 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:03.481+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:04.233 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:04.231+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:04.467 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:04.465+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:05.236 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:05.234+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:05.418 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:05.416+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:06.205 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:06.203+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:06.380 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:06.378+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:07.175 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:07.173+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:07.379 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:07.378+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:08.180 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:08.178+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:08.389 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:08.387+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:09.183 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:09.181+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:09.404 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:09.402+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:10.152 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:10.150+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:10.368 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:10.366+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:11.173 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:11.171+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:11.415 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:11.414+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:12.219 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:12.217+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:12.441 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:12.439+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:13.225 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:13.223+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:13.452 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:13.450+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:14.257 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:14.256+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:14.499 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:14.497+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:15.216 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:15.214+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:15.491 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:15.489+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:16.267 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:16.265+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:16.453 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:16.452+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:17.235 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:17.233+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:17.501 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:17.499+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:18.218 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:18.216+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:18.498 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:18.497+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:19.250 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:19.248+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:19.537 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:19.536+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:20.252 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:20.251+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:20.562 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:20.560+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:21.287 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:21.285+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:21.605 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:21.604+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:22.307 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:22.305+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:22.581 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:22.579+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:23.263 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:23.261+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:23.583 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:23.581+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:24.281 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:24.280+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:24.547 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:24.545+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:25.247 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:25.245+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:25.522 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:25.520+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:26.260 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:26.258+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:26.487 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:26.485+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:27.278 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:27.276+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:27.466 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:27.465+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:28.240 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:28.238+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:28.519 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:28.461+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:29.261 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:29.259+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:29.486 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:29.484+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:30.285 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:30.284+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:30.468 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:30.466+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:31.298 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:31.296+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:31.431 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:31.429+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:32.339 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:32.338+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:32.457 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:32.456+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:33.336 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:33.334+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:33.462 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:33.461+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:34.377 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:34.375+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:34.459 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:34.457+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:35.393 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:35.392+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:35.457 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:35.456+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:36.421 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:36.419+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:36.466 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:36.465+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:37.424 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:37.423+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:37.452 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:37.450+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:38.416 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:38.415+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:38.473 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:38.473+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:39.378 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:39.377+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:39.461 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:39.460+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:40.423 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:40.423+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:40.477 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:40.477+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:41.389 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:41.389+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:41.528 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:41.527+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:42.434 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:42.433+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:42.541 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:42.541+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:43.450 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:43.450+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:43.580 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:43.579+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:44.433 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:44.433+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:44.539 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:44.538+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:45.394 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:45.393+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:45.531 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:45.530+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:46.402 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:46.401+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:46.542 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:46.542+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:47.439 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:47.438+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:47.570 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:47.569+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:48.471 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:48.471+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:48.558 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:48.557+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:49.426 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:49.425+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:49.516 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:49.515+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:50.410 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:50.410+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:50.479 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:50.479+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:51.396 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:51.396+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:51.515 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:51.515+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:52.441 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:52.440+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:52.487 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:52.486+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:53.426 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:53.426+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:53.515 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:53.514+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:54.429 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:54.428+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:54.556 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:54.556+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:55.409 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:55.408+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:55.529 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:55.529+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:56.449 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:56.449+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:56.531 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:56.531+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:57.500 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:57.499+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:57.550 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:57.549+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:58.454 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:58.453+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:58.569 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:58.568+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:59.433 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:58:59.432+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:58:59.586 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:58:59.584+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:00.439 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:00.437+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:00.542 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:00.540+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:01.481 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:01.480+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:01.493 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:01.491+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:02.460 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:02.458+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:02.483 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:02.482+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:03.428 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:03.426+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:03.505 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:03.504+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:04.433 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:04.431+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:04.508 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:04.507+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:05.456 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:05.454+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:05.464 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:05.462+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:06.445 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:06.444+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:06.473 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:06.472+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:07.445 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:07.444+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:07.465 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:07.463+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:08.452 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:08.450+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:08.474 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:08.472+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:09.485 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:09.484+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:09.497 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:09.496+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:10.455 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:10.453+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:10.532 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:10.530+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:11.422 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:11.421+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:11.567 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:11.566+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:12.373 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:12.372+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:12.582 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:12.581+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:13.394 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:13.392+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:13.601 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:13.599+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:14.349 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:14.348+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:14.573 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:14.572+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:15.342 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:15.340+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:15.542 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:15.540+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:16.317 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:16.315+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:16.528 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:16.526+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:17.340 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:17.339+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:17.547 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:17.545+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:18.318 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:18.316+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:18.501 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:18.499+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:19.287 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:19.286+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:19.488 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:19.486+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:20.299 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:20.297+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:20.475 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:20.473+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:21.326 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:21.324+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:21.485 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:21.484+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:22.348 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:22.347+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:22.479 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:22.477+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:23.301 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:23.299+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:23.459 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:23.457+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:24.283 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:24.281+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:24.437 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:24.435+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:25.309 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:25.306+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:25.420 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:25.418+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:26.278 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:26.273+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:26.465 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:26.463+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:27.270 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:27.268+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:27.444 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:27.443+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:28.318 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:28.316+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:28.482 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:28.480+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:29.325 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:29.323+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:29.439 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:29.437+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:30.296 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:30.295+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:30.480 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:30.478+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:31.300 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:31.298+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:31.449 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:31.447+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:32.293 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:32.292+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:32.413 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:32.412+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:33.328 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:33.327+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:33.457 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:33.455+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:34.340 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:34.339+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:34.433 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:34.431+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:35.352 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:35.350+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:35.478 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:35.476+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:36.355 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:36.354+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:36.436 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:36.435+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:37.392 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:37.392+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:37.419 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:37.419+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:38.344 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:38.343+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:38.438 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:38.437+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:39.312 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:39.311+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:39.413 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:39.412+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:40.341 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:40.341+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:40.375 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:40.374+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:41.348 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:41.348+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:41.366 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:41.365+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:42.334 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:42.333+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:42.405 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:42.404+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:43.384 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:43.383+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:43.417 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:43.416+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:44.422 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:44.421+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:44.433 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:44.432+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:45.408 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:45.407+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:45.464 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:45.463+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:46.455 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:46.455+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:46.463 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:46.463+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:47.460 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:47.459+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:47.461 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:47.461+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:48.417 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:48.415+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:48.417 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:48.417+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:49.378 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:49.377+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:49.429 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:49.428+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:50.383 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:50.382+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:50.397 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:50.396+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:51.406 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:51.405+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:51.414 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:51.413+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:52.432 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:52.431+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:52.440 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:52.439+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:53.439 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:53.439+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:53.444 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:53.444+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:54.476 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:54.475+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:54.487 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:54.487+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:55.434 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:55.433+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:55.519 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:55.519+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:56.391 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:56.390+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:56.506 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:56.506+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:57.409 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:57.407+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:57.481 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:57.479+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:58.368 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:58.366+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:58.491 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:58.489+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:59.417 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-05T23:59:59.416+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-05T23:59:59.509 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-05T23:59:59.507+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:00.398 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:00.396+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:00.476 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:00.473+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:01.441 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:01.439+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:01.510 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:01.508+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:02.473 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:02.471+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:02.525 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:02.523+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:03.510 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:03.508+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:03.560 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:03.558+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:04.544 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:04.542+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:04.555 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:04.554+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:05.587 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:05.586+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:05.594 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:05.592+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:06.561 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:06.560+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:06.602 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:06.600+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:07.538 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:07.537+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:07.596 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:07.594+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:08.557 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:08.555+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:08.567 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:08.565+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:09.572 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:09.571+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:09.582 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:09.580+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:10.592 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:10.591+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:10.608 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:10.607+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:11.547 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:11.545+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:11.648 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:11.646+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:12.543 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:12.542+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:12.680 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:12.678+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:13.504 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:13.502+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:13.656 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:13.654+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:14.469 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:14.467+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:14.636 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:14.634+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:15.445 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:15.444+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:15.649 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:15.647+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:16.486 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:16.485+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:16.686 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:16.684+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:17.515 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:17.513+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:17.668 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:17.667+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:18.522 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:18.520+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:18.637 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:18.635+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:19.568 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:19.566+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:19.681 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:19.680+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:20.568 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:20.566+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:20.731 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:20.730+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:21.563 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:21.561+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:21.749 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:21.748+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:22.583 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:22.582+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:22.723 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:22.722+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:23.540 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:23.538+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:23.726 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:23.724+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:24.531 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:24.530+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:24.753 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:24.751+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:25.503 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:25.502+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:25.795 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:25.794+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:26.466 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:26.464+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:26.785 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:26.783+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:27.484 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:27.482+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:27.827 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:27.825+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:28.477 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:28.476+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:28.797 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:28.796+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:29.522 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:29.520+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:29.783 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:29.779+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:30.549 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:30.548+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:30.829 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:30.827+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:31.588 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:31.586+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:31.833 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:31.832+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:32.551 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:32.549+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:32.849 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:32.847+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:33.543 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:33.541+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:33.813 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:33.812+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:34.534 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:34.533+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:34.774 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:34.773+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:35.576 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:35.576+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:35.735 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:35.734+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:36.623 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:36.622+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:36.698 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:36.697+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:37.583 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:37.582+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:37.651 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:37.651+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:38.548 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:38.547+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:38.699 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:38.698+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:39.571 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:39.570+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:39.655 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:39.654+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:40.573 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:40.572+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:40.624 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:40.624+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:41.585 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:41.584+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:41.586 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:41.585+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:42.576 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:42.575+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:42.597 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:42.597+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:43.555 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:43.554+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:43.586 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:43.585+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:44.521 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:44.520+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:44.610 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:44.610+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:45.476 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:45.475+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:45.601 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:45.600+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:46.438 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:46.437+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:46.615 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:46.614+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:47.473 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:47.472+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:47.655 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:47.654+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:48.460 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:48.460+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:48.608 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:48.607+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:49.427 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:49.426+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:49.658 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:49.658+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:50.437 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:50.436+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:50.629 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:50.628+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:51.480 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:51.479+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:51.580 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:51.580+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:52.496 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:52.496+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:52.548 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:52.547+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:53.522 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:53.521+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:53.532 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:53.532+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:54.479 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:54.479+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:54.567 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:54.567+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:55.489 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:55.487+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:55.579 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:55.577+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:56.508 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:56.507+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:56.567 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:56.565+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:57.484 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:57.482+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:57.539 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:57.537+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:58.496 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:58.494+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:58.579 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:58.578+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:59.534 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:00:59.532+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:00:59.618 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:00:59.617+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:00.521 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:00.520+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:00.607 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:00.604+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:01.488 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:01.487+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:01.576 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:01.574+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:02.453 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:02.451+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:02.597 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:02.595+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:03.501 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:03.499+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:03.601 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:03.599+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:04.462 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:04.460+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:04.571 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:04.569+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:05.469 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:05.468+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:05.614 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:05.612+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:06.457 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:06.456+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:06.578 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:06.576+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:07.482 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:07.480+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:07.534 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:07.533+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:08.466 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:08.465+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:08.574 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:08.573+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:09.449 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:09.447+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:09.619 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:09.617+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:10.490 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:10.488+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:10.656 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:10.655+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:11.536 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:11.534+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:11.677 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:11.675+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:12.500 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:12.499+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:12.629 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:12.628+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:13.515 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:13.514+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:13.653 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:13.651+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:14.561 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:14.559+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:14.637 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:14.635+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:15.520 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:15.519+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:15.621 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:15.619+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:16.569 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:16.567+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:16.634 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:16.633+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:17.599 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:17.597+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:17.673 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:17.671+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:18.614 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:18.612+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:18.625 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:18.623+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:19.602 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:19.600+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:19.622 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:19.621+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:20.643 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:20.641+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:20.645 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:20.643+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:21.599 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:21.597+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:21.613 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:21.611+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:22.625 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:22.624+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:22.651 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:22.649+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:23.635 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:23.633+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:23.675 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:23.673+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:24.590 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:24.589+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:24.662 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:24.661+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:25.622 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:25.620+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:25.640 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:25.639+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:26.586 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:26.584+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:26.675 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:26.673+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:27.558 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:27.556+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:27.688 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:27.687+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:28.522 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:28.521+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:28.673 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:28.672+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:29.501 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:29.499+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:29.708 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:29.707+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:30.491 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:30.490+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:30.747 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:30.745+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:31.472 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:31.470+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:31.703 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:31.702+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:32.479 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:32.477+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:32.734 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:32.733+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:33.457 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:33.456+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:33.747 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:33.746+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:34.438 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:34.437+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:34.784 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:34.784+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:35.464 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:35.463+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:35.748 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:35.747+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:36.454 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:36.453+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:36.733 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:36.732+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:37.483 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:37.482+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:37.735 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:37.734+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:38.451 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:38.450+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:38.710 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:38.709+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:39.492 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:39.491+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:39.663 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:39.662+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:40.526 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:40.526+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:40.628 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:40.628+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:41.482 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:41.481+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:41.615 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:41.614+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:42.476 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:42.475+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:42.652 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:42.652+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:43.471 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:43.471+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:43.650 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:43.649+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:44.464 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:44.463+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:44.644 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:44.644+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:45.456 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:45.456+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:45.611 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:45.611+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:46.416 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:46.415+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:46.660 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:46.659+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:47.434 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:47.433+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:47.621 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:47.620+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:48.478 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:48.478+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:48.667 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:48.666+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:49.482 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:49.482+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:49.683 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:49.682+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:50.499 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:50.498+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:50.635 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:50.634+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:51.490 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:51.490+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:51.600 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:51.599+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:52.527 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:52.526+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:52.563 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:52.562+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:53.532 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:53.530+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:53.600 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:53.599+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:54.490 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:54.489+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:54.604 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:54.603+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:55.472 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:55.471+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:55.591 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:55.589+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:56.470 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:56.468+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:56.554 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:56.553+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:57.449 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:57.446+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:57.533 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:57.531+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:58.462 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:58.460+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:58.538 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:58.536+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:59.480 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:01:59.478+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:01:59.554 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:01:59.553+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:00.487 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:00.485+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:00.574 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:00.572+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:01.454 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:01.452+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:01.612 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:01.610+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:02.421 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:02.420+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:02.660 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:02.659+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:03.436 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:03.435+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:03.697 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:03.695+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:04.407 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:04.406+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:04.733 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:04.732+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:05.389 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:05.387+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:05.773 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:05.771+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:06.353 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:06.351+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:06.736 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:06.734+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:07.310 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:07.308+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:07.729 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:07.727+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:08.304 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:08.303+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:08.755 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:08.754+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:09.281 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:09.280+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:09.762 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:09.761+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:10.277 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:10.276+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:10.771 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:10.769+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:11.278 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:11.276+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:11.774 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:11.772+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:12.238 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:12.237+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:12.732 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:12.730+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:13.209 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:13.208+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:13.717 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:13.715+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:14.174 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:14.172+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:14.736 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:14.735+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:15.129 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:15.127+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:15.726 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:15.725+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:16.166 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:16.165+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:16.745 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:16.744+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:17.138 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:17.137+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:17.749 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:17.747+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:18.167 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:18.165+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:18.720 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:18.718+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:19.202 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:19.201+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:19.726 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:19.725+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:20.189 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:20.187+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:20.731 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:20.729+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:21.166 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:21.165+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:21.689 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:21.688+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:22.179 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:22.177+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:22.669 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:22.667+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:23.131 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:23.129+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:23.684 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:23.682+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:24.177 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:24.176+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:24.635 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:24.633+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:25.146 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:25.145+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:25.644 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:25.643+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:26.187 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:26.186+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:26.662 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:26.661+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:27.235 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:27.233+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:27.692 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:27.690+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:28.261 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:28.259+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:28.671 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:28.669+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:29.263 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:29.261+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:29.721 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:29.720+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:30.292 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:30.290+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:30.693 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:30.691+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:31.311 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:31.310+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:31.719 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:31.718+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:32.307 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:32.306+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:32.716 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:32.716+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:33.350 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:33.349+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:33.702 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:33.701+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:34.341 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:34.341+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:34.698 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:34.698+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:35.338 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:35.338+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:35.714 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:35.713+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:36.306 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:36.305+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:36.726 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:36.725+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:37.283 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:37.282+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:37.715 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:37.715+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:38.282 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:38.281+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:38.673 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:38.672+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:39.252 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:39.251+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:39.717 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:39.716+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:40.247 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:40.247+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:40.747 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:40.746+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:41.216 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:41.216+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:41.777 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:41.777+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:42.234 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:42.234+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:42.781 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:42.780+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:43.206 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:43.205+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:43.794 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:43.793+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:44.185 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:44.185+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:44.818 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:44.817+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:45.192 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:45.191+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:45.818 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:45.818+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:46.242 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:46.241+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:46.805 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:46.804+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:47.283 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:47.282+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:47.764 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:47.763+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:48.291 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:48.291+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:48.731 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:48.731+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:49.279 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:49.278+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:49.745 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:49.744+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:50.294 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:50.294+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:50.726 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:50.726+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:51.269 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:51.269+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:51.697 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:51.696+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:52.222 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:52.220+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:52.704 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:52.702+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:53.178 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:53.176+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:53.749 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:53.747+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:54.225 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:54.223+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:54.740 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:54.738+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:55.268 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:55.266+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:55.708 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:55.706+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:56.247 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:56.246+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:56.675 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:56.674+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:57.209 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:57.207+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:57.633 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:57.632+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:58.237 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:58.235+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:58.643 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:58.642+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:59.282 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:02:59.280+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:02:59.627 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:02:59.625+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:00.313 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:00.312+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:00.669 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:00.668+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:01.340 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:01.338+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:01.713 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:01.711+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:02.323 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:02.321+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:02.693 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:02.692+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:03.314 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:03.312+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:03.718 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:03.717+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:04.287 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:04.285+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:04.767 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:04.765+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:05.250 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:05.249+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:05.754 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:05.753+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:06.289 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:06.288+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:06.736 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:06.734+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:07.326 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:07.324+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:07.717 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:07.715+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:08.287 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:08.286+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:08.708 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:08.706+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:09.291 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:09.289+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:09.748 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:09.746+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:10.291 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:10.289+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:10.736 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:10.735+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:11.255 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:11.254+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:11.752 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:11.750+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:12.239 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:12.237+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:12.743 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:12.741+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:13.227 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:13.226+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:13.758 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:13.757+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:14.272 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:14.270+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:14.787 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:14.786+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:15.310 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:15.308+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:15.741 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:15.740+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:16.271 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:16.270+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:16.717 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:16.716+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:17.241 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:17.238+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:17.701 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:17.699+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:18.244 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:18.242+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:18.693 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:18.692+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:19.265 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:19.264+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:19.678 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:19.676+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:20.257 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:20.255+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:20.633 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:20.632+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:21.273 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:21.271+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:21.678 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:21.677+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:22.269 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:22.267+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:22.666 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:22.665+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:23.298 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:23.296+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:23.653 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:23.652+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:24.325 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:24.324+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:24.636 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:24.634+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:25.358 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:25.357+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:25.641 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:25.639+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:26.327 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:26.325+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:26.655 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:26.653+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:27.339 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:27.337+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:27.652 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:27.650+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:28.296 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:28.294+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:28.669 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:28.668+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:29.344 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:29.342+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:29.682 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:29.681+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:30.388 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:30.388+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:30.671 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:30.670+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:31.396 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:31.396+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:31.634 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:31.634+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:32.445 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:32.444+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:32.594 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:32.594+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:33.428 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:33.427+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:33.570 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:33.569+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:34.387 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:34.386+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:34.597 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:34.596+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:35.427 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:35.426+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:35.557 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:35.556+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:36.438 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:36.438+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:36.534 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:36.533+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:37.456 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:37.455+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:37.510 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:37.509+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:38.440 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:38.439+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:38.521 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:38.521+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:39.479 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:39.478+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:39.562 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:39.562+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:40.529 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:40.528+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:40.601 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:40.600+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:41.510 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:41.509+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:41.610 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:41.610+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:42.501 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:42.500+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:42.656 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:42.656+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:43.483 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:43.482+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:43.637 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:43.636+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:44.514 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:44.513+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:44.616 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:44.615+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:45.526 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:45.526+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:45.597 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:45.597+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:46.488 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:46.488+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:46.594 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:46.593+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:47.512 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:47.511+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:47.643 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:47.642+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:48.504 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:48.503+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:48.687 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:48.687+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:49.486 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:49.486+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:49.730 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:49.729+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:50.476 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:50.474+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:50.737 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:50.735+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:51.490 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:51.488+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:51.738 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:51.736+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:52.468 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:52.466+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:52.691 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:52.689+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:53.462 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:53.460+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:53.738 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:53.736+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:54.446 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:54.445+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:54.714 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:54.713+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:55.441 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:55.439+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:55.696 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:55.695+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:56.419 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:56.418+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:56.719 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:56.717+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:57.470 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:57.468+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:57.696 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:57.694+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:58.509 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:58.507+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:58.727 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:58.725+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:59.512 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:03:59.510+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:03:59.725 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:03:59.723+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:00.501 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:00.500+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:00.710 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:00.709+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:01.454 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:01.452+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:01.715 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:01.714+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:02.463 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:02.462+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:02.764 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:02.762+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:03.493 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:03.492+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:03.795 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:03.793+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:04.482 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:04.480+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:04.798 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:04.797+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:05.465 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:05.463+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:05.791 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:05.789+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:06.500 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:06.499+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:06.763 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:06.762+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:07.528 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:07.526+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:07.772 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:07.771+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:08.518 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:08.517+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:08.817 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:08.815+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:09.559 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:09.557+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:09.806 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:09.804+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:10.592 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:10.590+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:10.823 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:10.822+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:11.557 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:11.556+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:11.841 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:11.840+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:12.539 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:12.537+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:12.831 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:12.830+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:13.536 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:13.534+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:13.832 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:13.830+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:14.549 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:14.547+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:14.787 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:14.785+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:15.533 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:15.531+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:15.749 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:15.747+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:16.549 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:16.547+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:16.734 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:16.732+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:17.545 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:17.543+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:17.772 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:17.771+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:18.547 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:18.545+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:18.791 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:18.789+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:19.563 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:19.561+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:19.766 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:19.764+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:20.525 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:20.523+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:20.814 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:20.811+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:21.489 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:21.487+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:21.853 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:21.851+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:22.481 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:22.480+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:22.866 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:22.864+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:23.493 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:23.491+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:23.896 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:23.894+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:24.491 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:24.490+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:24.897 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:24.895+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:25.489 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:25.487+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:25.864 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:25.862+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:26.484 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:26.482+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:26.823 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:26.820+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:27.478 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:27.477+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:27.845 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:27.844+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:28.441 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:28.440+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:28.799 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:28.798+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:29.440 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:29.440+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:29.836 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:29.835+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:30.448 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:30.447+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:30.817 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:30.817+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:31.461 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:31.461+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:31.791 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:31.790+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:32.421 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:32.421+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:32.799 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:32.799+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:33.436 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:33.436+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:33.803 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:33.802+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:34.447 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:34.447+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:34.818 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:34.817+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:35.445 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:35.444+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:35.841 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:35.841+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:36.479 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:36.478+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:36.861 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:36.861+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:37.497 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:37.497+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:37.855 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:37.855+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:38.466 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:38.466+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:38.826 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:38.825+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:39.453 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:39.453+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:39.876 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:39.876+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:40.409 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:40.408+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:40.902 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:40.902+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:41.395 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:41.395+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:41.867 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:41.867+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:42.367 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:42.366+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:42.877 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:42.876+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:43.352 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:43.351+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:43.895 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:43.894+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:44.351 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:44.350+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:44.873 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:44.873+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:45.368 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:45.367+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:45.836 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:45.835+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:46.409 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:46.408+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:46.883 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:46.882+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:47.449 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:47.448+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:47.869 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:47.868+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:48.472 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:48.470+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:48.849 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:48.847+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:49.449 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:49.447+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:49.857 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:49.856+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:50.438 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:50.436+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:50.869 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:50.868+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:51.481 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:51.480+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:51.899 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:51.898+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:52.501 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:52.499+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:52.892 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:52.890+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:53.482 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:53.481+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:53.928 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:53.926+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:54.522 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:54.520+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:54.910 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:54.908+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:55.488 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:55.487+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:55.897 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:55.896+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:56.528 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:56.526+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:56.943 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:56.942+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:57.569 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:57.567+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:57.949 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:57.948+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:58.610 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:58.608+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:58.955 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:58.953+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:59.621 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:04:59.620+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:04:59.931 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:04:59.929+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:00.611 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:00.610+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:00.952 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:00.950+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:01.653 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:01.652+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:01.981 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:01.980+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:02.645 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:02.644+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:02.989 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:02.987+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:03.632 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:03.631+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:04.001 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:03.999+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:04.637 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:04.635+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:04.956 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:04.954+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:05.594 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:05.593+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:05.912 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:05.910+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:06.605 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:06.603+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:06.871 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:06.870+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:07.652 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:07.650+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:07.864 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:07.863+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:08.664 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:08.662+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:08.901 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:08.899+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:09.648 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:09.646+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:09.905 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:09.903+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:10.618 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:10.617+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:10.916 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:10.914+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:11.628 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:11.627+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:11.965 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:11.964+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:12.581 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:12.579+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:12.956 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:12.954+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:13.614 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:13.613+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:13.961 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:13.959+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:14.634 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:14.632+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:14.917 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:14.915+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:15.585 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:15.584+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:15.937 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:15.935+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:16.554 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:16.553+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:16.976 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:16.974+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:17.591 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:17.589+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:17.974 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:17.973+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:18.587 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:18.585+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:18.955 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:18.953+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:19.580 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:19.578+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:19.914 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:19.913+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:20.569 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:20.568+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:20.897 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:20.896+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:21.522 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:21.521+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:21.851 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:21.850+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:22.478 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:22.476+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:22.891 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:22.889+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:23.476 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:23.474+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:23.842 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:23.840+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:24.431 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:24.429+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:24.805 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:24.803+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:25.455 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:25.453+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:25.848 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:25.847+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:26.439 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:26.439+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:26.886 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:26.885+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:27.410 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:27.409+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:27.852 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:27.852+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:28.403 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:28.402+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:28.863 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:28.862+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:29.408 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:29.408+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:29.858 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:29.857+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:30.429 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:30.428+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:30.851 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:30.851+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:31.381 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:31.380+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:31.807 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:31.807+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:32.415 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:32.414+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:32.798 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:32.797+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:33.369 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:33.369+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:33.819 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:33.818+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:34.393 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:34.393+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:34.784 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:34.783+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:35.442 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:35.442+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:35.737 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:35.737+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:36.476 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:36.475+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:36.779 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:36.779+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:37.518 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:37.517+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:37.771 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:37.771+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:38.528 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:38.527+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:38.811 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:38.810+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:39.548 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:39.547+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:39.845 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:39.844+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:40.538 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:40.538+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:40.852 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:40.852+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:41.508 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:41.507+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:41.808 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:41.808+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:42.508 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:42.508+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:42.762 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:42.761+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:43.514 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:43.513+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:43.735 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:43.734+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:44.470 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:44.470+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:44.760 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:44.759+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:45.454 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:45.453+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:45.721 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:45.720+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:46.412 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:46.410+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:46.680 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:46.678+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:47.443 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:47.441+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:47.682 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:47.680+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:48.406 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:48.404+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:48.688 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:48.686+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:49.359 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:49.358+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:49.664 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:49.662+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:50.330 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:50.329+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:50.707 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:50.705+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:51.293 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:51.292+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:51.746 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:51.745+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:52.267 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:52.265+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:52.721 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:52.720+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:53.302 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:53.301+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:53.683 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:53.681+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:54.283 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:54.282+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:54.702 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:54.701+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:55.333 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:55.331+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:55.737 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:55.735+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:56.299 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:56.298+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:56.781 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:56.779+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:57.284 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:57.283+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:57.750 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:57.748+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:58.293 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:58.292+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:58.752 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:58.751+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:59.344 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:05:59.342+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:05:59.746 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:05:59.744+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:00.373 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:00.372+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:00.765 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:00.764+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:01.326 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:01.324+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:01.774 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:01.772+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:02.313 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:02.311+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:02.769 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:02.767+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:03.296 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:03.295+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:03.782 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:03.780+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:04.274 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:04.272+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:04.808 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:04.806+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:05.285 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:05.283+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:05.853 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:05.852+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:06.267 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:06.265+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:06.873 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:06.871+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:07.263 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:07.262+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:07.846 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:07.844+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:08.284 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:08.283+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:08.804 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:08.802+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:09.321 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:09.319+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:09.843 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:09.842+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:10.303 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:10.301+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:10.892 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:10.890+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:11.303 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:11.301+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:11.863 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:11.861+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:12.271 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:12.270+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:12.908 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:12.906+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:13.275 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:13.274+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:13.907 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:13.906+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:14.315 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:14.314+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:14.911 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:14.910+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:15.268 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:15.266+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:15.880 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:15.879+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:16.240 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:16.238+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:16.873 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:16.872+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:17.264 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:17.262+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:17.857 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:17.856+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:18.286 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:18.284+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:18.898 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:18.896+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:19.315 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:19.313+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:19.948 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:19.947+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:20.341 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:20.339+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:20.980 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:20.978+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:21.383 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:21.381+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:21.988 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:21.986+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:22.429 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:22.427+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:22.975 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:22.975+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:23.444 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:23.442+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:23.953 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:23.952+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:24.476 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:24.475+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:25.003 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:25.003+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:25.465 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:25.464+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:25.965 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:25.964+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:26.488 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:26.488+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:26.990 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:26.990+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:27.497 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:27.496+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:27.974 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:27.973+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:28.525 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:28.524+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:28.958 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:28.957+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:29.489 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:29.489+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:29.991 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:29.990+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:30.491 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:30.490+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:31.007 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:31.005+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:31.525 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:31.524+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:31.998 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:31.998+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:32.554 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:32.553+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:32.956 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:32.955+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:33.514 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:33.514+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:33.913 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:33.912+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:34.504 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:34.503+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:34.952 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:34.951+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:35.491 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:35.490+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:35.981 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:35.981+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:36.470 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:36.469+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:36.958 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:36.957+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:37.495 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:37.495+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:37.927 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:37.927+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:38.493 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:38.493+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:38.934 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:38.934+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:39.537 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:39.537+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:39.956 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:39.956+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:40.574 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:40.573+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:40.915 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:40.915+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:41.615 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:41.615+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:41.900 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:41.900+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:42.640 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:42.640+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:42.855 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:42.855+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:43.629 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:43.628+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:43.826 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:43.825+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:44.650 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:44.648+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:44.809 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:44.808+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:45.648 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:45.647+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:45.852 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:45.851+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:46.626 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:46.624+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:46.885 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:46.883+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:47.637 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:47.636+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:47.891 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:47.889+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:48.605 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:48.603+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:48.915 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:48.913+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:49.634 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:49.632+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:49.905 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:49.904+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:50.596 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:50.594+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:50.932 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:50.930+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:51.570 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:51.568+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:51.946 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:51.944+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:52.530 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:52.528+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:52.987 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:52.985+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:53.493 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:53.492+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:53.971 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:53.969+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:54.485 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:54.484+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:55.007 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:55.006+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:55.510 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:55.508+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:55.994 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:55.992+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:56.530 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:56.529+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:57.030 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:57.029+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:57.515 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:57.514+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:58.033 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:58.032+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:58.508 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:58.507+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:59.037 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:06:59.035+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:06:59.547 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:06:59.545+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:00.010 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:00.008+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:00.515 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:00.513+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:01.017 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:01.015+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:01.472 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:01.471+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:02.055 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:02.053+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:02.457 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:02.455+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:03.017 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:03.015+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:03.454 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:03.452+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:04.028 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:04.026+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:04.472 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:04.471+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:05.067 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:05.065+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:05.493 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:05.492+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:06.107 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:06.105+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:06.470 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:06.468+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:07.066 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:07.064+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:07.486 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:07.485+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:08.022 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:08.021+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:08.516 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:08.515+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:09.060 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:09.058+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:09.492 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:09.490+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:10.078 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:10.076+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:10.466 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:10.464+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:11.064 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:11.063+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:11.441 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:11.439+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:12.079 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:12.077+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:12.443 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:12.442+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:13.084 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:13.082+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:13.478 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:13.477+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:14.123 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:14.121+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:14.476 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:14.474+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:15.110 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:15.108+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:15.487 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:15.485+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:16.104 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:16.103+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:16.464 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:16.462+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:17.099 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:17.098+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:17.455 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:17.453+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:18.089 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:18.087+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:18.416 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:18.415+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:19.096 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:19.094+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:19.390 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:19.388+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:20.061 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:20.060+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:20.393 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:20.391+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:21.020 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:21.019+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:21.434 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:21.433+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:22.013 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:22.012+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:22.392 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:22.391+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:23.052 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:23.052+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:23.418 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:23.418+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:24.080 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:24.079+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:24.439 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:24.438+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:25.042 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:25.042+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:25.404 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:25.404+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:26.053 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:26.052+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:26.450 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:26.449+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:27.078 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:27.077+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:27.424 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:27.424+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:28.031 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:28.030+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:28.474 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:28.474+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:29.075 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:29.074+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:29.505 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:29.505+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:30.074 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:30.073+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:30.189 INFO:tasks.workunit.client.0.smithi167.stderr:2023-10-06T00:07:30.188+0000 7f991f7fe700 -1 received signal: Terminated from timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/cli_generic.sh (PID: 148952) UID: 1000 2023-10-06T00:07:30.190 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-10-06T00:07:30.504 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:30.503+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:31.060 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:31.060+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:31.514 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:31.513+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:32.100 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:32.100+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:32.485 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:32.484+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:33.100 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:33.100+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:33.528 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:33.527+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:34.117 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:34.116+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:34.496 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:34.496+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:35.152 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:35.152+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:35.521 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:35.521+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:36.109 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:36.109+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:36.567 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:36.566+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:37.120 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:37.119+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:37.608 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:37.608+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:38.106 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:38.105+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:38.615 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:38.615+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:39.128 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:39.127+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:39.583 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:39.582+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:40.125 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:40.124+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:40.624 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:40.624+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:41.156 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:41.155+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:41.592 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:41.592+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:42.110 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:42.109+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:42.606 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:42.605+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:43.139 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:43.137+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:43.652 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:43.650+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:44.146 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:44.145+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:44.691 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:44.689+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:45.218 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:45.135+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:45.642 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:45.641+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:46.184 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:46.182+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:46.636 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:46.634+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:47.172 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:47.170+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:47.621 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:47.619+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:48.171 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:48.169+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:48.654 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:48.653+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:49.220 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:49.219+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:49.682 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:49.680+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:50.177 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:50.175+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:50.636 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:50.635+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:51.214 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:51.212+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:51.638 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:51.636+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:52.190 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:52.188+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:52.594 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:52.592+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:53.204 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:53.202+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:53.628 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:53.627+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:54.231 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:54.229+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:54.658 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:54.656+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:55.208 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:55.207+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:55.655 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:55.653+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:56.188 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:56.186+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:56.612 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:56.611+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:57.142 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:57.141+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:57.651 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:57.650+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:58.121 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:58.119+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:58.623 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:58.622+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:59.091 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:07:59.090+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:07:59.668 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:07:59.666+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:00.115 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:00.113+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:00.619 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:00.618+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:01.075 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:01.074+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:01.626 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:01.624+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:02.116 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:02.114+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:02.615 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:02.613+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:03.143 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:03.141+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:03.646 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:03.645+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:04.157 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:04.155+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:04.666 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:04.665+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:05.138 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:05.136+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:05.676 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:05.674+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:06.121 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:06.119+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:06.671 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:06.669+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:07.110 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:07.109+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:07.637 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:07.635+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:08.090 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:08.089+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:08.651 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:08.650+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:09.102 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:09.101+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:09.696 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:09.694+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:10.091 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:10.089+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:10.672 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:10.671+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:11.114 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:11.112+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:11.643 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:11.641+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:12.110 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:12.109+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:12.638 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:12.637+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:13.158 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:13.156+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:13.652 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:13.650+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:14.140 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:14.138+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:14.657 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:14.655+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:15.154 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:15.152+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:15.651 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:15.650+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:16.204 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:16.203+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:16.662 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:16.660+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:17.160 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:17.158+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:17.619 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:17.618+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:18.115 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:18.113+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:18.629 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:18.628+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:19.158 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:19.156+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:19.586 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:19.584+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:20.187 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:20.185+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:20.540 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:20.539+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:21.152 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:21.152+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:21.540 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:21.540+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:22.159 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:22.158+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:22.536 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:22.535+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:23.168 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:23.167+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:23.577 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:23.577+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:24.212 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:24.211+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:24.536 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:24.535+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:25.204 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:25.204+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:25.525 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:25.524+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:26.170 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:26.169+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:26.554 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:26.553+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:27.164 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:27.164+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:27.564 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:27.563+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:28.136 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:28.135+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:28.598 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:28.598+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:29.123 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:29.122+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:29.633 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:29.632+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:30.097 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:30.096+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:30.191 DEBUG:teuthology.orchestra.run:timed out waiting; will kill: 2023-10-06T00:08:30.667 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:30.662+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:31.109 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:31.108+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:31.708 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:31.707+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:32.071 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:32.071+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:32.747 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:32.747+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:33.048 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:33.047+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:33.745 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:33.742+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:34.040 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:34.039+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:34.717 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:34.716+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:35.026 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:35.025+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:35.677 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:35.676+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:36.053 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:36.053+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:36.703 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:36.703+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:37.023 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:37.022+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:37.659 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:37.659+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:38.015 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:38.014+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:38.641 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:38.641+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:39.062 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:39.061+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:39.687 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:39.686+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:40.101 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:40.099+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:40.647 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:40.647+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:41.072 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:41.071+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:41.649 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:41.648+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:42.048 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:42.047+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:42.632 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:42.630+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:43.050 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:43.048+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:43.677 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:43.675+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:44.002 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:44.000+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:44.721 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:44.719+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:44.988 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:44.986+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:45.757 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:45.755+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:45.961 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:45.960+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:46.800 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:46.798+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:46.920 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:46.918+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:47.813 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:47.812+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:47.929 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:47.927+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:48.776 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:48.774+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:48.892 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:48.890+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:49.794 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:49.792+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:49.849 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:49.847+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:50.792 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:50.790+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:50.848 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:50.847+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:51.791 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:51.790+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:51.808 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:51.807+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:52.768 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:52.766+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:52.846 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:52.844+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:53.772 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:53.769+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:53.797 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:53.795+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:54.764 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:54.762+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:54.775 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:54.773+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:55.723 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:55.721+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:55.744 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:55.742+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:56.746 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:56.745+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:56.753 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:56.751+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:57.747 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:57.746+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:57.772 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:57.771+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:58.756 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:58.755+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:58.785 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:58.784+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:59.789 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:08:59.787+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:08:59.795 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:08:59.794+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:00.837 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:00.836+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:00.839 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:00.838+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:01.824 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:01.822+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:01.876 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:01.875+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:02.800 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:02.799+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:02.867 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:02.865+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:03.815 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:03.814+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:03.845 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:03.844+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:04.773 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:04.771+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:04.887 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:04.886+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:05.818 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:05.816+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:05.878 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:05.876+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:06.835 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:06.833+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:06.883 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:06.881+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:07.816 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:07.814+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:07.908 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:07.906+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:08.818 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:08.816+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:08.859 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:08.858+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:09.829 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:09.828+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:09.908 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:09.907+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:10.842 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:10.840+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:10.956 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:10.954+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:11.838 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:11.837+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:12.001 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:11.999+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:12.820 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:12.818+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:13.049 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:13.048+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:13.859 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:13.857+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:14.021 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:14.020+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:14.820 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:14.819+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:15.013 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:15.012+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:15.805 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:15.803+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:15.975 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:15.974+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:16.797 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:16.795+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:16.942 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:16.940+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:17.798 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:17.797+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:17.959 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:17.958+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:18.844 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:18.843+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:18.970 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:18.969+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:19.892 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:19.892+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:19.923 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:19.922+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:20.908 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:20.907+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:20.930 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:20.930+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:21.887 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:21.886+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:21.968 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:21.968+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:22.865 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:22.864+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:22.960 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:22.960+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:23.822 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:23.821+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:23.942 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:23.942+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:24.802 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:24.801+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:24.942 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:24.941+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:25.833 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:25.832+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:25.958 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:25.957+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:26.883 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:26.882+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:26.964 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:26.964+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:27.931 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:27.930+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:27.998 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:27.998+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:28.901 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:28.900+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:28.961 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:28.961+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:29.887 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:29.887+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:30.003 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:30.002+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:30.193 DEBUG:teuthology.orchestra.run:timed out waiting; will kill: 2023-10-06T00:09:30.193 INFO:tasks.workunit:Stopping ['rbd/cli_generic.sh'] on client.0... 2023-10-06T00:09:30.193 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-10-06T00:09:30.475 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_487ffcb5ddb60d29c5688d1a457e812b1b697b90/qa/tasks/workunit.py", line 129, in task p.spawn(_run_tests, ctx, refspec, role, tests, File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_487ffcb5ddb60d29c5688d1a457e812b1b697b90/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test rbd/cli_generic.sh) on smithi167 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=487ffcb5ddb60d29c5688d1a457e812b1b697b90 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/cli_generic.sh' 2023-10-06T00:09:30.645 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=0bdd485f18fc4b7199f1461fe7f6ee79 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_487ffcb5ddb60d29c5688d1a457e812b1b697b90/qa/tasks/workunit.py", line 129, in task p.spawn(_run_tests, ctx, refspec, role, tests, File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_487ffcb5ddb60d29c5688d1a457e812b1b697b90/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test rbd/cli_generic.sh) on smithi167 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=487ffcb5ddb60d29c5688d1a457e812b1b697b90 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/cli_generic.sh' 2023-10-06T00:09:30.648 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-06T00:09:30.658 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-10-06T00:09:30.658 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T00:09:30.914 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:30.914+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:30.979 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:30.978+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:31.037 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:09:31.037 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":160,"fsid":"18db5e25-e3f3-4284-9e58-5e5554ae5140","created":"2023-10-05T21:06:16.596517+0000","modified":"2023-10-05T22:09:47.050511+0000","last_up_change":"2023-10-05T21:06:27.614053+0000","last_in_change":"2023-10-05T21:06:17.629595+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-05T21:06:28.369471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-05T21:06:32.019894+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"18","last_force_op_resend_prenautilus":"18","last_force_op_resend_preluminous":"18","auid":0,"snap_mode":"selfmanaged","snap_seq":91,"snap_epoch":156,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[3],"tier_of":-1,"read_tier":3,"write_tier":3,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cache","create_time":"2023-10-05T21:06:39.346838+0000","flags":8201,"flags_names":"hashpspool,incomplete_clones,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"18","last_force_op_resend_prenautilus":"18","last_force_op_resend_preluminous":"18","auid":0,"snap_mode":"selfmanaged","snap_seq":91,"snap_epoch":156,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":2,"read_tier":-1,"write_tier":-1,"cache_mode":"writeback","target_max_bytes":0,"target_max_objects":250,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"bloom","false_positive_probability":0.050000000000000003,"target_size":0,"seed":0},"hit_set_period":60,"hit_set_count":8,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"ab74b673-a624-4ec2-ad02-409b63519337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":144114},{"type":"v1","addr":"172.21.15.167:6817","nonce":144114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":144114},{"type":"v1","addr":"172.21.15.167:6819","nonce":144114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":144114},{"type":"v1","addr":"172.21.15.167:6823","nonce":144114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":144114},{"type":"v1","addr":"172.21.15.167:6821","nonce":144114}]},"public_addr":"172.21.15.167:6817/144114","cluster_addr":"172.21.15.167:6819/144114","heartbeat_back_addr":"172.21.15.167:6823/144114","heartbeat_front_addr":"172.21.15.167:6821/144114","state":["exists","up"]},{"osd":1,"uuid":"77f471b1-fe86-4b52-8091-01e503bab3c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":144115},{"type":"v1","addr":"172.21.15.167:6809","nonce":144115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":144115},{"type":"v1","addr":"172.21.15.167:6811","nonce":144115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":144115},{"type":"v1","addr":"172.21.15.167:6815","nonce":144115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":144115},{"type":"v1","addr":"172.21.15.167:6813","nonce":144115}]},"public_addr":"172.21.15.167:6809/144115","cluster_addr":"172.21.15.167:6811/144115","heartbeat_back_addr":"172.21.15.167:6815/144115","heartbeat_front_addr":"172.21.15.167:6813/144115","state":["exists","up"]},{"osd":2,"uuid":"2e146899-f47c-4e3e-aeb0-737ab3b287d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":144105},{"type":"v1","addr":"172.21.15.167:6801","nonce":144105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":144105},{"type":"v1","addr":"172.21.15.167:6803","nonce":144105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":144105},{"type":"v1","addr":"172.21.15.167:6807","nonce":144105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":144105},{"type":"v1","addr":"172.21.15.167:6805","nonce":144105}]},"public_addr":"172.21.15.167:6801/144105","cluster_addr":"172.21.15.167:6803/144105","heartbeat_back_addr":"172.21.15.167:6807/144105","heartbeat_front_addr":"172.21.15.167:6805/144105","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-05T21:06:22.818703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-05T21:06:22.854292+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-05T21:06:22.794012+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":4,"snaps":[{"begin":4,"length":1}]},{"pool":5,"snaps":[{"begin":2,"length":1},{"begin":4,"length":1}]},{"pool":7,"snaps":[{"begin":2,"length":1},{"begin":4,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T00:09:31.051 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-06T00:09:31.051 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:09:31.390 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:09:31.392 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:09:31.404 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":5648,"stamp":"2023-10-06T00:09:30.048569+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24943,"ondisk_log_size":24943,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002029"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":1926,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050211+0000","last_change":"2023-10-05T21:11:08.661112+0000","last_active":"2023-10-05T22:09:12.050211+0000","last_peered":"2023-10-05T22:09:12.050211+0000","last_clean":"2023-10-05T22:09:12.050211+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-05T22:09:12.050211+0000","last_undegraded":"2023-10-05T22:09:12.050211+0000","last_fullsized":"2023-10-05T22:09:12.050211+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T00:45:46.698848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1829,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050119+0000","last_change":"2023-10-05T21:11:08.660937+0000","last_active":"2023-10-05T22:09:12.050119+0000","last_peered":"2023-10-05T22:09:12.050119+0000","last_clean":"2023-10-05T22:09:12.050119+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-05T22:09:12.050119+0000","last_undegraded":"2023-10-05T22:09:12.050119+0000","last_fullsized":"2023-10-05T22:09:12.050119+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:22.382648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"158'775","reported_seq":1644,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050024+0000","last_change":"2023-10-05T21:11:08.660871+0000","last_active":"2023-10-05T22:09:12.050024+0000","last_peered":"2023-10-05T22:09:12.050024+0000","last_clean":"2023-10-05T22:09:12.050024+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-05T22:09:12.050024+0000","last_undegraded":"2023-10-05T22:09:12.050024+0000","last_fullsized":"2023-10-05T22:09:12.050024+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:17.807335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"158'3217","reported_seq":8586,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050627+0000","last_change":"2023-10-05T21:11:08.651828+0000","last_active":"2023-10-05T22:09:12.050627+0000","last_peered":"2023-10-05T22:09:12.050627+0000","last_clean":"2023-10-05T22:09:12.050627+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-05T22:09:12.050627+0000","last_undegraded":"2023-10-05T22:09:12.050627+0000","last_fullsized":"2023-10-05T22:09:12.050627+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":3217,"log_dups_size":0,"ondisk_log_size":3217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:52:36.783122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"158'1282","reported_seq":2162,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.048845+0000","last_change":"2023-10-05T21:11:08.653936+0000","last_active":"2023-10-05T22:09:12.048845+0000","last_peered":"2023-10-05T22:09:12.048845+0000","last_clean":"2023-10-05T22:09:12.048845+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-05T22:09:12.048845+0000","last_undegraded":"2023-10-05T22:09:12.048845+0000","last_fullsized":"2023-10-05T22:09:12.048845+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1282,"log_dups_size":0,"ondisk_log_size":1282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:52.571985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11081,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049738+0000","last_change":"2023-10-05T21:11:08.661771+0000","last_active":"2023-10-05T22:09:12.049738+0000","last_peered":"2023-10-05T22:09:12.049738+0000","last_clean":"2023-10-05T22:09:12.049738+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-05T22:09:12.049738+0000","last_undegraded":"2023-10-05T22:09:12.049738+0000","last_fullsized":"2023-10-05T22:09:12.049738+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:30:11.617699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":281,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049561+0000","last_change":"2023-10-05T21:06:29.922199+0000","last_active":"2023-10-05T22:09:12.049561+0000","last_peered":"2023-10-05T22:09:12.049561+0000","last_clean":"2023-10-05T22:09:12.049561+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-05T22:09:12.049561+0000","last_undegraded":"2023-10-05T22:09:12.049561+0000","last_fullsized":"2023-10-05T22:09:12.049561+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:28.614917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:28.614917+0000","last_clean_scrub_stamp":"2023-10-05T21:06:28.614917+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:59:56.310688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2345,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049831+0000","last_change":"2023-10-05T21:11:08.659625+0000","last_active":"2023-10-05T22:09:12.049831+0000","last_peered":"2023-10-05T22:09:12.049831+0000","last_clean":"2023-10-05T22:09:12.049831+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-05T22:09:12.049831+0000","last_undegraded":"2023-10-05T22:09:12.049831+0000","last_fullsized":"2023-10-05T22:09:12.049831+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-06T22:30:43.667801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11729,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049612+0000","last_change":"2023-10-05T21:11:08.661262+0000","last_active":"2023-10-05T22:09:12.049612+0000","last_peered":"2023-10-05T22:09:12.049612+0000","last_clean":"2023-10-05T22:09:12.049612+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-05T22:09:12.049612+0000","last_undegraded":"2023-10-05T22:09:12.049612+0000","last_fullsized":"2023-10-05T22:09:12.049612+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-06T21:11:58.444833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"158'884","reported_seq":1851,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.048835+0000","last_change":"2023-10-05T21:11:08.655946+0000","last_active":"2023-10-05T22:09:12.048835+0000","last_peered":"2023-10-05T22:09:12.048835+0000","last_clean":"2023-10-05T22:09:12.048835+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-05T22:09:12.048835+0000","last_undegraded":"2023-10-05T22:09:12.048835+0000","last_fullsized":"2023-10-05T22:09:12.048835+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:20:06.851803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11611,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050624+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-05T22:09:12.050624+0000","last_peered":"2023-10-05T22:09:12.050624+0000","last_clean":"2023-10-05T22:09:12.050624+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-05T22:09:12.050624+0000","last_undegraded":"2023-10-05T22:09:12.050624+0000","last_fullsized":"2023-10-05T22:09:12.050624+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-06T22:30:11.666110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1781,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050624+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-05T22:09:12.050624+0000","last_peered":"2023-10-05T22:09:12.050624+0000","last_clean":"2023-10-05T22:09:12.050624+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-05T22:09:12.050624+0000","last_undegraded":"2023-10-05T22:09:12.050624+0000","last_fullsized":"2023-10-05T22:09:12.050624+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T01:49:48.848559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1750,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049927+0000","last_change":"2023-10-05T21:11:08.661368+0000","last_active":"2023-10-05T22:09:12.049927+0000","last_peered":"2023-10-05T22:09:12.049927+0000","last_clean":"2023-10-05T22:09:12.049927+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-05T22:09:12.049927+0000","last_undegraded":"2023-10-05T22:09:12.049927+0000","last_fullsized":"2023-10-05T22:09:12.049927+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T01:16:04.809466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17193,"ondisk_log_size":17193,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7653,"ondisk_log_size":7653,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740573,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:08:40 2023","interfaces":[{"interface":"back","average":{"1min":0.22900000000000001,"5min":0.217,"15min":0.216},"min":{"1min":0.17599999999999999,"5min":0.151,"15min":0.14399999999999999},"max":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.19400000000000001},{"interface":"front","average":{"1min":0.25600000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"min":{"1min":0.21099999999999999,"5min":0.185,"15min":0.16700000000000001},"max":{"1min":0.44700000000000001,"5min":0.46800000000000003,"15min":0.47299999999999998},"last":0.218}]},{"osd":1,"last update":"Fri Oct 6 00:08:40 2023","interfaces":[{"interface":"back","average":{"1min":0.25600000000000001,"5min":0.23699999999999999,"15min":0.23899999999999999},"min":{"1min":0.188,"5min":0.183,"15min":0.153},"max":{"1min":0.434,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.27800000000000002,"5min":0.26700000000000002,"15min":0.26900000000000002},"min":{"1min":0.19900000000000001,"5min":0.16300000000000001,"15min":0.16300000000000001},"max":{"1min":0.48399999999999999,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.251}]}]},{"osd":0,"up_from":8,"seq":34359740575,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:08:52 2023","interfaces":[{"interface":"back","average":{"1min":0.20599999999999999,"5min":0.20899999999999999,"15min":0.21199999999999999},"min":{"1min":0.154,"5min":0.14099999999999999,"15min":0.13300000000000001},"max":{"1min":0.34499999999999997,"5min":0.41299999999999998,"15min":0.46100000000000002},"last":0.17599999999999999},{"interface":"front","average":{"1min":0.26800000000000002,"5min":0.26300000000000001,"15min":0.26400000000000001},"min":{"1min":0.23699999999999999,"5min":0.152,"15min":0.152},"max":{"1min":0.33400000000000002,"5min":0.47899999999999998,"15min":0.48399999999999999},"last":0.26700000000000002}]},{"osd":2,"last update":"Fri Oct 6 00:08:52 2023","interfaces":[{"interface":"back","average":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"min":{"1min":0.20200000000000001,"5min":0.17499999999999999,"15min":0.16500000000000001},"max":{"1min":0.30199999999999999,"5min":0.439,"15min":0.439},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.30099999999999999,"5min":0.29799999999999999,"15min":0.29599999999999999},"min":{"1min":0.26400000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.36299999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.29599999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740570,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.20000000000000001,"5min":0.20599999999999999,"15min":0.20999999999999999},"min":{"1min":0.14999999999999999,"5min":0.14299999999999999,"15min":0.13},"max":{"1min":0.30299999999999999,"5min":0.40999999999999998,"15min":0.42799999999999999},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.27200000000000002,"5min":0.27400000000000002,"15min":0.27900000000000003},"min":{"1min":0.22700000000000001,"5min":0.161,"15min":0.161},"max":{"1min":0.39900000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.24099999999999999}]},{"osd":2,"last update":"Fri Oct 6 00:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.22800000000000001,"5min":0.23200000000000001,"15min":0.23799999999999999},"min":{"1min":0.19400000000000001,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.33200000000000002,"5min":0.44,"15min":0.51500000000000001},"last":0.192},{"interface":"front","average":{"1min":0.26700000000000002,"5min":0.27100000000000002,"15min":0.27500000000000002},"min":{"1min":0.22500000000000001,"5min":0.19900000000000001,"15min":0.19500000000000001},"max":{"1min":0.36699999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.217}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:09:31.406 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:09:31.780 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:09:31.782 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:09:31.794 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":5648,"stamp":"2023-10-06T00:09:30.048569+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24943,"ondisk_log_size":24943,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002029"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":1926,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050211+0000","last_change":"2023-10-05T21:11:08.661112+0000","last_active":"2023-10-05T22:09:12.050211+0000","last_peered":"2023-10-05T22:09:12.050211+0000","last_clean":"2023-10-05T22:09:12.050211+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-05T22:09:12.050211+0000","last_undegraded":"2023-10-05T22:09:12.050211+0000","last_fullsized":"2023-10-05T22:09:12.050211+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T00:45:46.698848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1829,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050119+0000","last_change":"2023-10-05T21:11:08.660937+0000","last_active":"2023-10-05T22:09:12.050119+0000","last_peered":"2023-10-05T22:09:12.050119+0000","last_clean":"2023-10-05T22:09:12.050119+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-05T22:09:12.050119+0000","last_undegraded":"2023-10-05T22:09:12.050119+0000","last_fullsized":"2023-10-05T22:09:12.050119+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:22.382648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"158'775","reported_seq":1644,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050024+0000","last_change":"2023-10-05T21:11:08.660871+0000","last_active":"2023-10-05T22:09:12.050024+0000","last_peered":"2023-10-05T22:09:12.050024+0000","last_clean":"2023-10-05T22:09:12.050024+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-05T22:09:12.050024+0000","last_undegraded":"2023-10-05T22:09:12.050024+0000","last_fullsized":"2023-10-05T22:09:12.050024+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:17.807335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"158'3217","reported_seq":8586,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050627+0000","last_change":"2023-10-05T21:11:08.651828+0000","last_active":"2023-10-05T22:09:12.050627+0000","last_peered":"2023-10-05T22:09:12.050627+0000","last_clean":"2023-10-05T22:09:12.050627+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-05T22:09:12.050627+0000","last_undegraded":"2023-10-05T22:09:12.050627+0000","last_fullsized":"2023-10-05T22:09:12.050627+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":3217,"log_dups_size":0,"ondisk_log_size":3217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:52:36.783122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"158'1282","reported_seq":2162,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.048845+0000","last_change":"2023-10-05T21:11:08.653936+0000","last_active":"2023-10-05T22:09:12.048845+0000","last_peered":"2023-10-05T22:09:12.048845+0000","last_clean":"2023-10-05T22:09:12.048845+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-05T22:09:12.048845+0000","last_undegraded":"2023-10-05T22:09:12.048845+0000","last_fullsized":"2023-10-05T22:09:12.048845+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1282,"log_dups_size":0,"ondisk_log_size":1282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:52.571985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11081,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049738+0000","last_change":"2023-10-05T21:11:08.661771+0000","last_active":"2023-10-05T22:09:12.049738+0000","last_peered":"2023-10-05T22:09:12.049738+0000","last_clean":"2023-10-05T22:09:12.049738+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-05T22:09:12.049738+0000","last_undegraded":"2023-10-05T22:09:12.049738+0000","last_fullsized":"2023-10-05T22:09:12.049738+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:30:11.617699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":281,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049561+0000","last_change":"2023-10-05T21:06:29.922199+0000","last_active":"2023-10-05T22:09:12.049561+0000","last_peered":"2023-10-05T22:09:12.049561+0000","last_clean":"2023-10-05T22:09:12.049561+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-05T22:09:12.049561+0000","last_undegraded":"2023-10-05T22:09:12.049561+0000","last_fullsized":"2023-10-05T22:09:12.049561+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:28.614917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:28.614917+0000","last_clean_scrub_stamp":"2023-10-05T21:06:28.614917+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:59:56.310688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2345,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049831+0000","last_change":"2023-10-05T21:11:08.659625+0000","last_active":"2023-10-05T22:09:12.049831+0000","last_peered":"2023-10-05T22:09:12.049831+0000","last_clean":"2023-10-05T22:09:12.049831+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-05T22:09:12.049831+0000","last_undegraded":"2023-10-05T22:09:12.049831+0000","last_fullsized":"2023-10-05T22:09:12.049831+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-06T22:30:43.667801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11729,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049612+0000","last_change":"2023-10-05T21:11:08.661262+0000","last_active":"2023-10-05T22:09:12.049612+0000","last_peered":"2023-10-05T22:09:12.049612+0000","last_clean":"2023-10-05T22:09:12.049612+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-05T22:09:12.049612+0000","last_undegraded":"2023-10-05T22:09:12.049612+0000","last_fullsized":"2023-10-05T22:09:12.049612+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-06T21:11:58.444833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"158'884","reported_seq":1851,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.048835+0000","last_change":"2023-10-05T21:11:08.655946+0000","last_active":"2023-10-05T22:09:12.048835+0000","last_peered":"2023-10-05T22:09:12.048835+0000","last_clean":"2023-10-05T22:09:12.048835+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-05T22:09:12.048835+0000","last_undegraded":"2023-10-05T22:09:12.048835+0000","last_fullsized":"2023-10-05T22:09:12.048835+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:20:06.851803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11611,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050624+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-05T22:09:12.050624+0000","last_peered":"2023-10-05T22:09:12.050624+0000","last_clean":"2023-10-05T22:09:12.050624+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-05T22:09:12.050624+0000","last_undegraded":"2023-10-05T22:09:12.050624+0000","last_fullsized":"2023-10-05T22:09:12.050624+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-06T22:30:11.666110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1781,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050624+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-05T22:09:12.050624+0000","last_peered":"2023-10-05T22:09:12.050624+0000","last_clean":"2023-10-05T22:09:12.050624+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-05T22:09:12.050624+0000","last_undegraded":"2023-10-05T22:09:12.050624+0000","last_fullsized":"2023-10-05T22:09:12.050624+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T01:49:48.848559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1750,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049927+0000","last_change":"2023-10-05T21:11:08.661368+0000","last_active":"2023-10-05T22:09:12.049927+0000","last_peered":"2023-10-05T22:09:12.049927+0000","last_clean":"2023-10-05T22:09:12.049927+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-05T22:09:12.049927+0000","last_undegraded":"2023-10-05T22:09:12.049927+0000","last_fullsized":"2023-10-05T22:09:12.049927+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T01:16:04.809466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17193,"ondisk_log_size":17193,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7653,"ondisk_log_size":7653,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740573,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:08:40 2023","interfaces":[{"interface":"back","average":{"1min":0.22900000000000001,"5min":0.217,"15min":0.216},"min":{"1min":0.17599999999999999,"5min":0.151,"15min":0.14399999999999999},"max":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.19400000000000001},{"interface":"front","average":{"1min":0.25600000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"min":{"1min":0.21099999999999999,"5min":0.185,"15min":0.16700000000000001},"max":{"1min":0.44700000000000001,"5min":0.46800000000000003,"15min":0.47299999999999998},"last":0.218}]},{"osd":1,"last update":"Fri Oct 6 00:08:40 2023","interfaces":[{"interface":"back","average":{"1min":0.25600000000000001,"5min":0.23699999999999999,"15min":0.23899999999999999},"min":{"1min":0.188,"5min":0.183,"15min":0.153},"max":{"1min":0.434,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.27800000000000002,"5min":0.26700000000000002,"15min":0.26900000000000002},"min":{"1min":0.19900000000000001,"5min":0.16300000000000001,"15min":0.16300000000000001},"max":{"1min":0.48399999999999999,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.251}]}]},{"osd":0,"up_from":8,"seq":34359740575,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:08:52 2023","interfaces":[{"interface":"back","average":{"1min":0.20599999999999999,"5min":0.20899999999999999,"15min":0.21199999999999999},"min":{"1min":0.154,"5min":0.14099999999999999,"15min":0.13300000000000001},"max":{"1min":0.34499999999999997,"5min":0.41299999999999998,"15min":0.46100000000000002},"last":0.17599999999999999},{"interface":"front","average":{"1min":0.26800000000000002,"5min":0.26300000000000001,"15min":0.26400000000000001},"min":{"1min":0.23699999999999999,"5min":0.152,"15min":0.152},"max":{"1min":0.33400000000000002,"5min":0.47899999999999998,"15min":0.48399999999999999},"last":0.26700000000000002}]},{"osd":2,"last update":"Fri Oct 6 00:08:52 2023","interfaces":[{"interface":"back","average":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"min":{"1min":0.20200000000000001,"5min":0.17499999999999999,"15min":0.16500000000000001},"max":{"1min":0.30199999999999999,"5min":0.439,"15min":0.439},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.30099999999999999,"5min":0.29799999999999999,"15min":0.29599999999999999},"min":{"1min":0.26400000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.36299999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.29599999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740570,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.20000000000000001,"5min":0.20599999999999999,"15min":0.20999999999999999},"min":{"1min":0.14999999999999999,"5min":0.14299999999999999,"15min":0.13},"max":{"1min":0.30299999999999999,"5min":0.40999999999999998,"15min":0.42799999999999999},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.27200000000000002,"5min":0.27400000000000002,"15min":0.27900000000000003},"min":{"1min":0.22700000000000001,"5min":0.161,"15min":0.161},"max":{"1min":0.39900000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.24099999999999999}]},{"osd":2,"last update":"Fri Oct 6 00:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.22800000000000001,"5min":0.23200000000000001,"15min":0.23799999999999999},"min":{"1min":0.19400000000000001,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.33200000000000002,"5min":0.44,"15min":0.51500000000000001},"last":0.192},{"interface":"front","average":{"1min":0.26700000000000002,"5min":0.27100000000000002,"15min":0.27500000000000002},"min":{"1min":0.22500000000000001,"5min":0.19900000000000001,"15min":0.19500000000000001},"max":{"1min":0.36699999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.217}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:09:31.795 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-06T00:09:31.795 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:09:31.914 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:31.914+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:32.006 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:32.006+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:32.134 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:09:32.136 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:09:32.149 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":5649,"stamp":"2023-10-06T00:09:32.049067+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24943,"ondisk_log_size":24943,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002079"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":1926,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050211+0000","last_change":"2023-10-05T21:11:08.661112+0000","last_active":"2023-10-05T22:09:12.050211+0000","last_peered":"2023-10-05T22:09:12.050211+0000","last_clean":"2023-10-05T22:09:12.050211+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-05T22:09:12.050211+0000","last_undegraded":"2023-10-05T22:09:12.050211+0000","last_fullsized":"2023-10-05T22:09:12.050211+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T00:45:46.698848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1829,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050119+0000","last_change":"2023-10-05T21:11:08.660937+0000","last_active":"2023-10-05T22:09:12.050119+0000","last_peered":"2023-10-05T22:09:12.050119+0000","last_clean":"2023-10-05T22:09:12.050119+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-05T22:09:12.050119+0000","last_undegraded":"2023-10-05T22:09:12.050119+0000","last_fullsized":"2023-10-05T22:09:12.050119+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:22.382648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"158'775","reported_seq":1644,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050024+0000","last_change":"2023-10-05T21:11:08.660871+0000","last_active":"2023-10-05T22:09:12.050024+0000","last_peered":"2023-10-05T22:09:12.050024+0000","last_clean":"2023-10-05T22:09:12.050024+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-05T22:09:12.050024+0000","last_undegraded":"2023-10-05T22:09:12.050024+0000","last_fullsized":"2023-10-05T22:09:12.050024+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:17.807335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"158'3217","reported_seq":8586,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050627+0000","last_change":"2023-10-05T21:11:08.651828+0000","last_active":"2023-10-05T22:09:12.050627+0000","last_peered":"2023-10-05T22:09:12.050627+0000","last_clean":"2023-10-05T22:09:12.050627+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-05T22:09:12.050627+0000","last_undegraded":"2023-10-05T22:09:12.050627+0000","last_fullsized":"2023-10-05T22:09:12.050627+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":3217,"log_dups_size":0,"ondisk_log_size":3217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:52:36.783122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"158'1282","reported_seq":2162,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.048845+0000","last_change":"2023-10-05T21:11:08.653936+0000","last_active":"2023-10-05T22:09:12.048845+0000","last_peered":"2023-10-05T22:09:12.048845+0000","last_clean":"2023-10-05T22:09:12.048845+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-05T22:09:12.048845+0000","last_undegraded":"2023-10-05T22:09:12.048845+0000","last_fullsized":"2023-10-05T22:09:12.048845+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1282,"log_dups_size":0,"ondisk_log_size":1282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:52.571985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11081,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049738+0000","last_change":"2023-10-05T21:11:08.661771+0000","last_active":"2023-10-05T22:09:12.049738+0000","last_peered":"2023-10-05T22:09:12.049738+0000","last_clean":"2023-10-05T22:09:12.049738+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-05T22:09:12.049738+0000","last_undegraded":"2023-10-05T22:09:12.049738+0000","last_fullsized":"2023-10-05T22:09:12.049738+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:30:11.617699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":281,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049561+0000","last_change":"2023-10-05T21:06:29.922199+0000","last_active":"2023-10-05T22:09:12.049561+0000","last_peered":"2023-10-05T22:09:12.049561+0000","last_clean":"2023-10-05T22:09:12.049561+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-05T22:09:12.049561+0000","last_undegraded":"2023-10-05T22:09:12.049561+0000","last_fullsized":"2023-10-05T22:09:12.049561+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:28.614917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:28.614917+0000","last_clean_scrub_stamp":"2023-10-05T21:06:28.614917+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:59:56.310688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2345,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049831+0000","last_change":"2023-10-05T21:11:08.659625+0000","last_active":"2023-10-05T22:09:12.049831+0000","last_peered":"2023-10-05T22:09:12.049831+0000","last_clean":"2023-10-05T22:09:12.049831+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-05T22:09:12.049831+0000","last_undegraded":"2023-10-05T22:09:12.049831+0000","last_fullsized":"2023-10-05T22:09:12.049831+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-06T22:30:43.667801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11729,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049612+0000","last_change":"2023-10-05T21:11:08.661262+0000","last_active":"2023-10-05T22:09:12.049612+0000","last_peered":"2023-10-05T22:09:12.049612+0000","last_clean":"2023-10-05T22:09:12.049612+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-05T22:09:12.049612+0000","last_undegraded":"2023-10-05T22:09:12.049612+0000","last_fullsized":"2023-10-05T22:09:12.049612+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-06T21:11:58.444833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"158'884","reported_seq":1851,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.048835+0000","last_change":"2023-10-05T21:11:08.655946+0000","last_active":"2023-10-05T22:09:12.048835+0000","last_peered":"2023-10-05T22:09:12.048835+0000","last_clean":"2023-10-05T22:09:12.048835+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-05T22:09:12.048835+0000","last_undegraded":"2023-10-05T22:09:12.048835+0000","last_fullsized":"2023-10-05T22:09:12.048835+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:20:06.851803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11611,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050624+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-05T22:09:12.050624+0000","last_peered":"2023-10-05T22:09:12.050624+0000","last_clean":"2023-10-05T22:09:12.050624+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-05T22:09:12.050624+0000","last_undegraded":"2023-10-05T22:09:12.050624+0000","last_fullsized":"2023-10-05T22:09:12.050624+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-06T22:30:11.666110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1781,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050624+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-05T22:09:12.050624+0000","last_peered":"2023-10-05T22:09:12.050624+0000","last_clean":"2023-10-05T22:09:12.050624+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-05T22:09:12.050624+0000","last_undegraded":"2023-10-05T22:09:12.050624+0000","last_fullsized":"2023-10-05T22:09:12.050624+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T01:49:48.848559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1750,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049927+0000","last_change":"2023-10-05T21:11:08.661368+0000","last_active":"2023-10-05T22:09:12.049927+0000","last_peered":"2023-10-05T22:09:12.049927+0000","last_clean":"2023-10-05T22:09:12.049927+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-05T22:09:12.049927+0000","last_undegraded":"2023-10-05T22:09:12.049927+0000","last_fullsized":"2023-10-05T22:09:12.049927+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T01:16:04.809466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17193,"ondisk_log_size":17193,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7653,"ondisk_log_size":7653,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740573,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:08:40 2023","interfaces":[{"interface":"back","average":{"1min":0.22900000000000001,"5min":0.217,"15min":0.216},"min":{"1min":0.17599999999999999,"5min":0.151,"15min":0.14399999999999999},"max":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.19400000000000001},{"interface":"front","average":{"1min":0.25600000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"min":{"1min":0.21099999999999999,"5min":0.185,"15min":0.16700000000000001},"max":{"1min":0.44700000000000001,"5min":0.46800000000000003,"15min":0.47299999999999998},"last":0.218}]},{"osd":1,"last update":"Fri Oct 6 00:08:40 2023","interfaces":[{"interface":"back","average":{"1min":0.25600000000000001,"5min":0.23699999999999999,"15min":0.23899999999999999},"min":{"1min":0.188,"5min":0.183,"15min":0.153},"max":{"1min":0.434,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.27800000000000002,"5min":0.26700000000000002,"15min":0.26900000000000002},"min":{"1min":0.19900000000000001,"5min":0.16300000000000001,"15min":0.16300000000000001},"max":{"1min":0.48399999999999999,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.251}]}]},{"osd":0,"up_from":8,"seq":34359740575,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:08:52 2023","interfaces":[{"interface":"back","average":{"1min":0.20599999999999999,"5min":0.20899999999999999,"15min":0.21199999999999999},"min":{"1min":0.154,"5min":0.14099999999999999,"15min":0.13300000000000001},"max":{"1min":0.34499999999999997,"5min":0.41299999999999998,"15min":0.46100000000000002},"last":0.17599999999999999},{"interface":"front","average":{"1min":0.26800000000000002,"5min":0.26300000000000001,"15min":0.26400000000000001},"min":{"1min":0.23699999999999999,"5min":0.152,"15min":0.152},"max":{"1min":0.33400000000000002,"5min":0.47899999999999998,"15min":0.48399999999999999},"last":0.26700000000000002}]},{"osd":2,"last update":"Fri Oct 6 00:08:52 2023","interfaces":[{"interface":"back","average":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"min":{"1min":0.20200000000000001,"5min":0.17499999999999999,"15min":0.16500000000000001},"max":{"1min":0.30199999999999999,"5min":0.439,"15min":0.439},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.30099999999999999,"5min":0.29799999999999999,"15min":0.29599999999999999},"min":{"1min":0.26400000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.36299999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.29599999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740570,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.20000000000000001,"5min":0.20599999999999999,"15min":0.20999999999999999},"min":{"1min":0.14999999999999999,"5min":0.14299999999999999,"15min":0.13},"max":{"1min":0.30299999999999999,"5min":0.40999999999999998,"15min":0.42799999999999999},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.27200000000000002,"5min":0.27400000000000002,"15min":0.27900000000000003},"min":{"1min":0.22700000000000001,"5min":0.161,"15min":0.161},"max":{"1min":0.39900000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.24099999999999999}]},{"osd":2,"last update":"Fri Oct 6 00:09:18 2023","interfaces":[{"interface":"back","average":{"1min":0.22800000000000001,"5min":0.23200000000000001,"15min":0.23799999999999999},"min":{"1min":0.19400000000000001,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.33200000000000002,"5min":0.44,"15min":0.51500000000000001},"last":0.192},{"interface":"front","average":{"1min":0.26700000000000002,"5min":0.27100000000000002,"15min":0.27500000000000002},"min":{"1min":0.22500000000000001,"5min":0.19900000000000001,"15min":0.19500000000000001},"max":{"1min":0.36699999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.217}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:09:32.150 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T00:09:32.513 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:09:32.514 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":160,"fsid":"18db5e25-e3f3-4284-9e58-5e5554ae5140","created":"2023-10-05T21:06:16.596517+0000","modified":"2023-10-05T22:09:47.050511+0000","last_up_change":"2023-10-05T21:06:27.614053+0000","last_in_change":"2023-10-05T21:06:17.629595+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-05T21:06:28.369471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-05T21:06:32.019894+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"18","last_force_op_resend_prenautilus":"18","last_force_op_resend_preluminous":"18","auid":0,"snap_mode":"selfmanaged","snap_seq":91,"snap_epoch":156,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[3],"tier_of":-1,"read_tier":3,"write_tier":3,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cache","create_time":"2023-10-05T21:06:39.346838+0000","flags":8201,"flags_names":"hashpspool,incomplete_clones,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"18","last_force_op_resend_prenautilus":"18","last_force_op_resend_preluminous":"18","auid":0,"snap_mode":"selfmanaged","snap_seq":91,"snap_epoch":156,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":2,"read_tier":-1,"write_tier":-1,"cache_mode":"writeback","target_max_bytes":0,"target_max_objects":250,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"bloom","false_positive_probability":0.050000000000000003,"target_size":0,"seed":0},"hit_set_period":60,"hit_set_count":8,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"ab74b673-a624-4ec2-ad02-409b63519337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":144114},{"type":"v1","addr":"172.21.15.167:6817","nonce":144114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":144114},{"type":"v1","addr":"172.21.15.167:6819","nonce":144114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":144114},{"type":"v1","addr":"172.21.15.167:6823","nonce":144114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":144114},{"type":"v1","addr":"172.21.15.167:6821","nonce":144114}]},"public_addr":"172.21.15.167:6817/144114","cluster_addr":"172.21.15.167:6819/144114","heartbeat_back_addr":"172.21.15.167:6823/144114","heartbeat_front_addr":"172.21.15.167:6821/144114","state":["exists","up"]},{"osd":1,"uuid":"77f471b1-fe86-4b52-8091-01e503bab3c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":144115},{"type":"v1","addr":"172.21.15.167:6809","nonce":144115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":144115},{"type":"v1","addr":"172.21.15.167:6811","nonce":144115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":144115},{"type":"v1","addr":"172.21.15.167:6815","nonce":144115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":144115},{"type":"v1","addr":"172.21.15.167:6813","nonce":144115}]},"public_addr":"172.21.15.167:6809/144115","cluster_addr":"172.21.15.167:6811/144115","heartbeat_back_addr":"172.21.15.167:6815/144115","heartbeat_front_addr":"172.21.15.167:6813/144115","state":["exists","up"]},{"osd":2,"uuid":"2e146899-f47c-4e3e-aeb0-737ab3b287d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":144105},{"type":"v1","addr":"172.21.15.167:6801","nonce":144105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":144105},{"type":"v1","addr":"172.21.15.167:6803","nonce":144105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":144105},{"type":"v1","addr":"172.21.15.167:6807","nonce":144105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":144105},{"type":"v1","addr":"172.21.15.167:6805","nonce":144105}]},"public_addr":"172.21.15.167:6801/144105","cluster_addr":"172.21.15.167:6803/144105","heartbeat_back_addr":"172.21.15.167:6807/144105","heartbeat_front_addr":"172.21.15.167:6805/144105","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-05T21:06:22.818703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-05T21:06:22.854292+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-05T21:06:22.794012+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":4,"snaps":[{"begin":4,"length":1}]},{"pool":5,"snaps":[{"begin":2,"length":1},{"begin":4,"length":1}]},{"pool":7,"snaps":[{"begin":2,"length":1},{"begin":4,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T00:09:32.953 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:32.952+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:33.024 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:33.023+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:33.530 INFO:tasks.ceph:Scrubbing osd.0 2023-10-06T00:09:33.531 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-06T00:09:33.958 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:33.958+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:34.012 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:34.011+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:34.942 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:34.942+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:34.970 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:34.970+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:35.914 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:35.914+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:35.935 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:35.935+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:36.885 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:36.884+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:36.915 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:36.915+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:37.905 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:37.904+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:37.942 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:37.941+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:38.871 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:38.870+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:38.989 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:38.988+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:39.839 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:39.838+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:40.022 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:40.021+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:40.815 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:40.813+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:41.043 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:41.042+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:41.788 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:41.786+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:42.088 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:42.086+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:42.798 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:42.796+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:43.074 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:43.073+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:43.772 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:43.770+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:44.041 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:44.040+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:44.766 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:44.765+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:45.033 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:45.031+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:45.758 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:45.756+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:46.010 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:46.009+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:46.784 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:46.783+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:47.047 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:47.045+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:47.775 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:47.773+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:48.071 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:48.069+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:48.757 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:48.755+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:49.073 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:49.071+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:49.766 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:49.764+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:50.068 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:50.066+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:50.788 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:50.786+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:51.074 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:51.073+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:51.761 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:51.759+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:52.063 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:52.061+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:52.725 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:52.723+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:53.026 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:53.024+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:53.758 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:53.757+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:54.000 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:53.998+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:54.809 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:54.807+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:54.999 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:54.997+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:55.767 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:55.765+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:56.007 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:56.006+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:56.787 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:56.785+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:57.041 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:57.040+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:57.825 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:57.823+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:58.003 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:58.001+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:58.812 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:58.811+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:59.050 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:09:59.048+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:09:59.841 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:09:59.839+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:00.040 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:00.039+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:00.797 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:00.795+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:01.046 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:01.044+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:01.830 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:01.828+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:02.033 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:02.032+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:02.799 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:02.797+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:03.026 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:03.025+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:03.800 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:03.798+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:04.062 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:04.061+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:04.778 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:04.775+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:05.082 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:05.081+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:05.801 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:05.800+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:06.116 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:06.114+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:06.833 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:06.832+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:07.083 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:07.082+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:07.877 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:07.876+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:08.056 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:08.054+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:08.854 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:08.852+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:09.013 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:09.010+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:09.836 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:09.834+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:10.039 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:10.038+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:10.820 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:10.819+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:11.037 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:11.035+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:11.835 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:11.833+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:12.056 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:12.055+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:12.865 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:12.861+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:13.021 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:13.020+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:13.815 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:13.813+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:14.045 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:14.043+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:14.845 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:14.844+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:15.091 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:15.090+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:15.841 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:15.839+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:16.129 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:16.128+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:16.820 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:16.819+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:17.129 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:17.129+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:17.777 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:17.776+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:18.142 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:18.142+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:18.785 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:18.784+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:19.112 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:19.112+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:19.816 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:19.815+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:20.098 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:20.098+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:20.839 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:20.839+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:21.055 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:21.055+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:21.826 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:21.826+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:22.017 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:22.016+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:22.800 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:22.799+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:22.993 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:22.992+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:23.780 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:23.779+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:24.016 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:24.015+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:24.749 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:24.748+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:24.989 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:24.989+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:25.764 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:25.763+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:26.015 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:26.015+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:26.776 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:26.775+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:26.975 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:26.974+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:27.761 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:27.760+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:27.946 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:27.946+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:28.751 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:28.750+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:28.928 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:28.928+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:29.767 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:29.767+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:29.904 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:29.904+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:30.732 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:30.731+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:30.908 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:30.908+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:31.754 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:31.754+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:31.920 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:31.919+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:32.746 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:32.745+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:32.953 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:32.953+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:33.795 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:33.794+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:33.955 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:33.955+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:34.836 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:34.836+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:35.002 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:35.001+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:35.863 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:35.862+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:35.971 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:35.970+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:36.887 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:36.886+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:36.935 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:36.934+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:37.844 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:37.842+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:37.935 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:37.933+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:38.831 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:38.829+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:38.965 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:38.963+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:39.795 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:39.794+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:39.988 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:39.986+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:40.792 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:40.790+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:41.007 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:41.006+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:41.781 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:41.780+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:42.049 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:42.047+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:42.755 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:42.754+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:43.049 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:43.047+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:43.769 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:43.767+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:44.003 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:44.002+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:44.720 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:44.719+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:45.016 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:45.015+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:45.721 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:45.719+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:46.029 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:46.028+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:46.681 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:46.679+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:47.002 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:47.000+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:47.716 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:47.715+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:47.956 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:47.954+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:48.757 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:48.756+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:48.932 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:48.929+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:49.783 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:49.781+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:49.907 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:49.906+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:50.826 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:50.824+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:50.903 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:50.901+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:51.779 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:51.777+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:51.934 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:51.933+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:52.778 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:52.776+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:52.903 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:52.902+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:53.775 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:53.774+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:53.894 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:53.892+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:54.734 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:54.732+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:54.909 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:54.908+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:55.770 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:55.769+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:55.953 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:55.952+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:56.807 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:56.805+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:56.958 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:56.956+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:57.825 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:57.824+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:57.912 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:57.911+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:58.862 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:58.860+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:58.905 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:58.903+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:59.813 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:10:59.811+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:10:59.897 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:10:59.895+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:00.792 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:00.790+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:00.946 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:00.944+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:01.837 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:01.835+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:01.923 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:01.921+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:02.793 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:02.791+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:02.937 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:02.935+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:03.801 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:03.800+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:03.939 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:03.938+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:04.849 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:04.848+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:04.895 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:04.894+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:05.853 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:05.851+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:05.872 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:05.871+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:06.849 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:06.847+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:06.920 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:06.918+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:07.822 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:07.821+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:07.894 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:07.893+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:08.856 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:08.854+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:08.872 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:08.870+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:09.871 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:09.870+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:09.907 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:09.905+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:10.849 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:10.848+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:10.957 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:10.956+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:11.869 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:11.867+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:11.998 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:11.996+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:12.870 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:12.868+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:13.004 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:13.002+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:13.823 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:13.822+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:14.025 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:14.023+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:14.838 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:14.837+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:15.008 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:15.007+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:15.857 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:15.856+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:16.018 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:16.017+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:16.816 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:16.816+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:16.979 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:16.978+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:17.846 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:17.846+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:17.937 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:17.936+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:18.870 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:18.870+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:18.933 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:18.932+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:19.842 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:19.841+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:19.896 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:19.895+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:20.849 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:20.848+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:20.850 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:20.849+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:21.819 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:21.818+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:21.885 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:21.884+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:22.858 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:22.857+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:22.887 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:22.886+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:23.828 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:23.827+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:23.873 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:23.872+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:24.786 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:24.785+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:24.869 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:24.869+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:25.802 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:25.802+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:25.841 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:25.840+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:26.787 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:26.786+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:26.851 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:26.850+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:27.806 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:27.805+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:27.897 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:27.896+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:28.781 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:28.780+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:28.850 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:28.850+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:29.741 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:29.740+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:29.837 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:29.836+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:30.790 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:30.790+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:30.874 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:30.874+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:31.833 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:31.832+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:31.851 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:31.851+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:32.798 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:32.798+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:32.849 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:32.848+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:33.565 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-10-06T00:11:33.566 INFO:tasks.ceph:Scrubbing osd.1 2023-10-06T00:11:33.566 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-06T00:11:33.780 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:33.779+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:33.814 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:33.814+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:34.733 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:34.732+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:34.831 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:34.830+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:35.731 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:35.729+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:35.834 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:35.833+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:36.761 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:36.759+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:36.879 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:36.877+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:37.769 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:37.767+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:37.883 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:37.882+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:38.734 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:38.732+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:38.876 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:38.874+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:39.735 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:39.733+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:39.860 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:39.858+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:40.764 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:40.762+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:40.829 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:40.827+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:41.770 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:41.768+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:41.868 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:41.866+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:42.768 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:42.766+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:42.885 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:42.883+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:43.758 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:43.757+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:43.884 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:43.883+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:44.741 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:44.739+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:44.920 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:44.919+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:45.737 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:45.735+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:45.962 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:45.960+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:46.778 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:46.776+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:46.924 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:46.922+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:47.736 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:47.735+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:47.920 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:47.919+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:48.777 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:48.776+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:48.913 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:48.911+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:49.795 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:49.793+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:49.925 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:49.923+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:50.768 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:50.767+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:50.907 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:50.905+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:51.730 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:51.729+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:51.934 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:51.933+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:52.764 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:52.762+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:52.913 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:52.911+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:53.746 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:53.745+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:53.868 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:53.866+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:54.772 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:54.771+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:54.898 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:54.896+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:55.738 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:55.737+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:55.894 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:55.893+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:56.702 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:56.701+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:56.847 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:56.845+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:57.682 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:57.681+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:57.810 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:57.808+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:58.703 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:58.701+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:58.804 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:58.802+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:59.661 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:11:59.659+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:11:59.824 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:11:59.823+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:00.659 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:00.658+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:00.828 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:00.827+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:01.642 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:01.641+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:01.838 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:01.836+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:02.603 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:02.601+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:02.873 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:02.871+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:03.625 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:03.623+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:03.838 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:03.835+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:04.576 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:04.575+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:04.876 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:04.875+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:05.540 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:05.539+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:05.910 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:05.909+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:06.544 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:06.542+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:06.909 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:06.908+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:07.574 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:07.572+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:07.872 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:07.870+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:08.573 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:08.571+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:08.911 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:08.909+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:09.592 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:09.591+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:09.910 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:09.908+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:10.560 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:10.559+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:10.948 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:10.947+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:11.571 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:11.570+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:11.996 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:11.995+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:12.534 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:12.532+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:12.991 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:12.990+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:13.540 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:13.540+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:13.994 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:13.994+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:14.518 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:14.518+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:15.019 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:15.018+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:15.537 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:15.537+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:16.020 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:16.020+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:16.548 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:16.547+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:17.058 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:17.057+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:17.531 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:17.530+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:18.068 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:18.067+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:18.519 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:18.518+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:19.109 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:19.109+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:19.535 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:19.534+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:20.117 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:20.116+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:20.543 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:20.542+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:21.078 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:21.078+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:21.531 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:21.530+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:22.102 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:22.102+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:22.494 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:22.493+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:23.114 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:23.114+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:23.502 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:23.502+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:24.121 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:24.120+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:24.514 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:24.514+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:25.144 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:25.144+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:25.470 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:25.469+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:26.172 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:26.171+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:26.515 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:26.514+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:27.148 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:27.147+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:27.500 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:27.499+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:28.197 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:28.196+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:28.469 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:28.468+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:29.198 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:29.198+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:29.512 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:29.511+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:30.244 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:30.243+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:30.529 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:30.528+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:31.251 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:31.250+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:31.545 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:31.544+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:32.208 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:32.207+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:32.553 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:32.552+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:33.228 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:33.227+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:33.523 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:33.521+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:34.205 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:34.204+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:34.544 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:34.543+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:35.226 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:35.225+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:35.550 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:35.549+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:36.254 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:36.252+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:36.526 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:36.524+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:37.276 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:37.275+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:37.576 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:37.575+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:38.242 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:38.240+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:38.529 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:38.527+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:39.221 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:39.219+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:39.545 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:39.543+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:40.247 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:40.245+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:40.584 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:40.583+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:41.198 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:41.197+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:41.562 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:41.561+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:42.153 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:42.152+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:42.588 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:42.587+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:43.135 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:43.134+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:43.613 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:43.611+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:44.137 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:44.135+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:44.651 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:44.649+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:45.104 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:45.102+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:45.664 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:45.663+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:46.117 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:46.115+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:46.696 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:46.694+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:47.095 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:47.094+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:47.696 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:47.694+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:48.142 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:48.141+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:48.676 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:48.674+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:49.113 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:49.111+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:49.638 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:49.637+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:50.159 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:50.157+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:50.653 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:50.651+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:51.170 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:51.169+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:51.691 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:51.689+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:52.160 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:52.158+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:52.688 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:52.687+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:53.131 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:53.129+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:53.645 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:53.644+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:54.171 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:54.169+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:54.682 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:54.680+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:55.151 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:55.149+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:55.686 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:55.684+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:56.135 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:56.133+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:56.659 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:56.657+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:57.137 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:57.135+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:57.631 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:57.630+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:58.153 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:58.151+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:58.607 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:58.605+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:59.200 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:12:59.198+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:12:59.562 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:12:59.561+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:00.202 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:00.201+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:00.565 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:00.563+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:01.224 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:01.222+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:01.564 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:01.563+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:02.270 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:02.269+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:02.586 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:02.585+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:03.261 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:03.259+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:03.572 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:03.570+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:04.291 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:04.289+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:04.610 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:04.609+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:05.252 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:05.251+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:05.624 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:05.623+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:06.261 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:06.259+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:06.610 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:06.608+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:07.282 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:07.281+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:07.585 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:07.583+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:08.264 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:08.263+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:08.599 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:08.598+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:09.270 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:09.269+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:09.645 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:09.644+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:10.248 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:10.247+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:10.674 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:10.673+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:11.208 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:11.207+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:11.661 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:11.661+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:12.178 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:12.177+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:12.654 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:12.653+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:13.193 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:13.193+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:13.673 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:13.673+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:14.234 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:14.233+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:14.674 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:14.674+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:15.232 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:15.232+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:15.724 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:15.723+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:16.279 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:16.278+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:16.698 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:16.697+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:17.258 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:17.258+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:17.661 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:17.660+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:18.246 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:18.246+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:18.659 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:18.658+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:19.207 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:19.207+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:19.645 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:19.644+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:20.223 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:20.222+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:20.669 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:20.668+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:21.174 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:21.174+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:21.628 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:21.627+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:22.171 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:22.170+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:22.592 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:22.592+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:23.136 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:23.135+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:23.546 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:23.545+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:24.123 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:24.123+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:24.567 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:24.566+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:25.101 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:25.101+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:25.617 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:25.617+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:26.119 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:26.118+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:26.645 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:26.645+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:27.126 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:27.125+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:27.667 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:27.667+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:28.128 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:28.127+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:28.653 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:28.653+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:29.173 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:29.172+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:29.668 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:29.667+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:30.162 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:30.161+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:30.705 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:30.705+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:31.148 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:31.147+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:31.716 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:31.714+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:32.166 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:32.164+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:32.753 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:32.752+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:33.151 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:33.149+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:33.602 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-10-06T00:13:33.603 INFO:tasks.ceph:Scrubbing osd.2 2023-10-06T00:13:33.604 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-06T00:13:33.754 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2023-10-06T00:13:33.755 INFO:teuthology.orchestra.run.smithi167.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T00:13:33.755 INFO:teuthology.orchestra.run.smithi167.stdout:} 2023-10-06T00:13:33.764 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:33.762+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:33.768 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-06T00:13:34.140 INFO:teuthology.orchestra.run.smithi167.stderr:instructed osd(s) 2 to deep-scrub 2023-10-06T00:13:34.151 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:13:34.172 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:34.170+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:34.475 INFO:teuthology.orchestra.run.smithi167.stderr:2023-10-06T00:13:34.474+0000 7fc612ffd700 0 --2- 172.21.15.167:0/2056922276 >> v2:172.21.15.167:3300/0 conn(0x7fc61c091040 0x7fc61c12b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2023-10-06T00:13:34.719 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:13:34.721 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:13:34.733 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":5770,"stamp":"2023-10-06T00:13:34.085264+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24943,"ondisk_log_size":24943,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002003"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":1926,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050211+0000","last_change":"2023-10-05T21:11:08.661112+0000","last_active":"2023-10-05T22:09:12.050211+0000","last_peered":"2023-10-05T22:09:12.050211+0000","last_clean":"2023-10-05T22:09:12.050211+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-05T22:09:12.050211+0000","last_undegraded":"2023-10-05T22:09:12.050211+0000","last_fullsized":"2023-10-05T22:09:12.050211+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T00:45:46.698848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1829,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050119+0000","last_change":"2023-10-05T21:11:08.660937+0000","last_active":"2023-10-05T22:09:12.050119+0000","last_peered":"2023-10-05T22:09:12.050119+0000","last_clean":"2023-10-05T22:09:12.050119+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-05T22:09:12.050119+0000","last_undegraded":"2023-10-05T22:09:12.050119+0000","last_fullsized":"2023-10-05T22:09:12.050119+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:22.382648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"158'775","reported_seq":1644,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050024+0000","last_change":"2023-10-05T21:11:08.660871+0000","last_active":"2023-10-05T22:09:12.050024+0000","last_peered":"2023-10-05T22:09:12.050024+0000","last_clean":"2023-10-05T22:09:12.050024+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-05T22:09:12.050024+0000","last_undegraded":"2023-10-05T22:09:12.050024+0000","last_fullsized":"2023-10-05T22:09:12.050024+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:17.807335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"158'3217","reported_seq":8586,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050627+0000","last_change":"2023-10-05T21:11:08.651828+0000","last_active":"2023-10-05T22:09:12.050627+0000","last_peered":"2023-10-05T22:09:12.050627+0000","last_clean":"2023-10-05T22:09:12.050627+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-05T22:09:12.050627+0000","last_undegraded":"2023-10-05T22:09:12.050627+0000","last_fullsized":"2023-10-05T22:09:12.050627+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":3217,"log_dups_size":0,"ondisk_log_size":3217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:52:36.783122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"158'1282","reported_seq":2162,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.048845+0000","last_change":"2023-10-05T21:11:08.653936+0000","last_active":"2023-10-05T22:09:12.048845+0000","last_peered":"2023-10-05T22:09:12.048845+0000","last_clean":"2023-10-05T22:09:12.048845+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-05T22:09:12.048845+0000","last_undegraded":"2023-10-05T22:09:12.048845+0000","last_fullsized":"2023-10-05T22:09:12.048845+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1282,"log_dups_size":0,"ondisk_log_size":1282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:52.571985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11081,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049738+0000","last_change":"2023-10-05T21:11:08.661771+0000","last_active":"2023-10-05T22:09:12.049738+0000","last_peered":"2023-10-05T22:09:12.049738+0000","last_clean":"2023-10-05T22:09:12.049738+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-05T22:09:12.049738+0000","last_undegraded":"2023-10-05T22:09:12.049738+0000","last_fullsized":"2023-10-05T22:09:12.049738+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:30:11.617699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":281,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049561+0000","last_change":"2023-10-05T21:06:29.922199+0000","last_active":"2023-10-05T22:09:12.049561+0000","last_peered":"2023-10-05T22:09:12.049561+0000","last_clean":"2023-10-05T22:09:12.049561+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-05T22:09:12.049561+0000","last_undegraded":"2023-10-05T22:09:12.049561+0000","last_fullsized":"2023-10-05T22:09:12.049561+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:28.614917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:28.614917+0000","last_clean_scrub_stamp":"2023-10-05T21:06:28.614917+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:59:56.310688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2345,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049831+0000","last_change":"2023-10-05T21:11:08.659625+0000","last_active":"2023-10-05T22:09:12.049831+0000","last_peered":"2023-10-05T22:09:12.049831+0000","last_clean":"2023-10-05T22:09:12.049831+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-05T22:09:12.049831+0000","last_undegraded":"2023-10-05T22:09:12.049831+0000","last_fullsized":"2023-10-05T22:09:12.049831+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-06T22:30:43.667801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11729,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049612+0000","last_change":"2023-10-05T21:11:08.661262+0000","last_active":"2023-10-05T22:09:12.049612+0000","last_peered":"2023-10-05T22:09:12.049612+0000","last_clean":"2023-10-05T22:09:12.049612+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-05T22:09:12.049612+0000","last_undegraded":"2023-10-05T22:09:12.049612+0000","last_fullsized":"2023-10-05T22:09:12.049612+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-06T21:11:58.444833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"158'884","reported_seq":1851,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.048835+0000","last_change":"2023-10-05T21:11:08.655946+0000","last_active":"2023-10-05T22:09:12.048835+0000","last_peered":"2023-10-05T22:09:12.048835+0000","last_clean":"2023-10-05T22:09:12.048835+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-05T22:09:12.048835+0000","last_undegraded":"2023-10-05T22:09:12.048835+0000","last_fullsized":"2023-10-05T22:09:12.048835+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:20:06.851803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11611,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050624+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-05T22:09:12.050624+0000","last_peered":"2023-10-05T22:09:12.050624+0000","last_clean":"2023-10-05T22:09:12.050624+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-05T22:09:12.050624+0000","last_undegraded":"2023-10-05T22:09:12.050624+0000","last_fullsized":"2023-10-05T22:09:12.050624+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-06T22:30:11.666110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1781,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050624+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-05T22:09:12.050624+0000","last_peered":"2023-10-05T22:09:12.050624+0000","last_clean":"2023-10-05T22:09:12.050624+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-05T22:09:12.050624+0000","last_undegraded":"2023-10-05T22:09:12.050624+0000","last_fullsized":"2023-10-05T22:09:12.050624+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T01:49:48.848559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1750,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049927+0000","last_change":"2023-10-05T21:11:08.661368+0000","last_active":"2023-10-05T22:09:12.049927+0000","last_peered":"2023-10-05T22:09:12.049927+0000","last_clean":"2023-10-05T22:09:12.049927+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-05T22:09:12.049927+0000","last_undegraded":"2023-10-05T22:09:12.049927+0000","last_fullsized":"2023-10-05T22:09:12.049927+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T01:16:04.809466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17193,"ondisk_log_size":17193,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7653,"ondisk_log_size":7653,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740622,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.21199999999999999,"5min":0.222,"15min":0.218},"min":{"1min":0.16600000000000001,"5min":0.16200000000000001,"15min":0.14399999999999999},"max":{"1min":0.25800000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.157},{"interface":"front","average":{"1min":0.23100000000000001,"5min":0.247,"15min":0.245},"min":{"1min":0.187,"5min":0.18099999999999999,"15min":0.16700000000000001},"max":{"1min":0.29799999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.21199999999999999}]},{"osd":1,"last update":"Fri Oct 6 00:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.249,"5min":0.251,"15min":0.24299999999999999},"min":{"1min":0.17299999999999999,"5min":0.17299999999999999,"15min":0.153},"max":{"1min":0.30599999999999999,"5min":0.46700000000000003,"15min":0.48699999999999999},"last":0.20399999999999999},{"interface":"front","average":{"1min":0.26400000000000001,"5min":0.27500000000000002,"15min":0.27100000000000002},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.16300000000000001},"max":{"1min":0.313,"5min":0.48399999999999999,"15min":0.51200000000000001},"last":0.249}]}]},{"osd":0,"up_from":8,"seq":34359740624,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.183,"5min":0.188,"15min":0.20200000000000001},"min":{"1min":0.14299999999999999,"5min":0.128,"15min":0.128},"max":{"1min":0.23799999999999999,"5min":0.34799999999999998,"15min":0.439},"last":0.159},{"interface":"front","average":{"1min":0.26400000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"min":{"1min":0.20699999999999999,"5min":0.193,"15min":0.152},"max":{"1min":0.34899999999999998,"5min":0.40300000000000002,"15min":0.47899999999999998},"last":0.27000000000000002}]},{"osd":2,"last update":"Fri Oct 6 00:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.24199999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"min":{"1min":0.17499999999999999,"5min":0.16300000000000001,"15min":0.16300000000000001},"max":{"1min":0.32100000000000001,"5min":0.36599999999999999,"15min":0.439},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.29999999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"min":{"1min":0.23300000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.378,"5min":0.436,"15min":0.53600000000000003},"last":0.30199999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740619,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.21299999999999999,"5min":0.20100000000000001,"15min":0.20499999999999999},"min":{"1min":0.13100000000000001,"5min":0.13100000000000001,"15min":0.13},"max":{"1min":0.38400000000000001,"5min":0.38700000000000001,"15min":0.40999999999999998},"last":0.217},{"interface":"front","average":{"1min":0.27600000000000002,"5min":0.26800000000000002,"15min":0.27600000000000002},"min":{"1min":0.17599999999999999,"5min":0.16500000000000001,"15min":0.161},"max":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.51000000000000001},"last":0.20699999999999999}]},{"osd":2,"last update":"Fri Oct 6 00:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.246,"5min":0.22900000000000001,"15min":0.23699999999999999},"min":{"1min":0.20200000000000001,"5min":0.17499999999999999,"15min":0.17199999999999999},"max":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.51500000000000001},"last":0.17100000000000001},{"interface":"front","average":{"1min":0.28799999999999998,"5min":0.27000000000000002,"15min":0.27600000000000002},"min":{"1min":0.22600000000000001,"5min":0.20100000000000001,"15min":0.19900000000000001},"max":{"1min":0.439,"5min":0.439,"15min":0.48099999999999998},"last":0.23200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:13:34.737 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:13:34.738 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:13:34.738 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:13:34.738 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:13:34.738 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:13:34.738 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:13:34.739 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-05T21:06:28.614917+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=28, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:13:34.739 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:13:34.739 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:13:34.739 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:13:34.739 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:13:34.740 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:13:34.740 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:13:34.740 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:13:34.797 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:34.795+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:35.152 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:35.150+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:35.789 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:35.788+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:36.104 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:36.103+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:36.804 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:36.803+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:37.113 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:37.112+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:37.855 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:37.853+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:38.124 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:38.122+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:38.905 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:38.902+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:39.154 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:39.152+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:39.894 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:39.892+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:40.202 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:40.200+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:40.895 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:40.894+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:41.165 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:41.163+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:41.855 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:41.854+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:42.193 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:42.191+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:42.850 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:42.848+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:43.212 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:43.210+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:43.833 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:43.831+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:44.231 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:44.229+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:44.818 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:44.816+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:45.241 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:45.240+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:45.793 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:45.792+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:46.274 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:46.273+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:46.813 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:46.812+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:47.260 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:47.259+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:47.863 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:47.862+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:48.231 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:48.230+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:48.875 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:48.874+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:49.263 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:49.255+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:50.333 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:49.892+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:50.334 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:50.224+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:50.918 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:50.916+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:51.193 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:51.192+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:51.911 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:51.910+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:52.207 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:52.205+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:52.936 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:52.934+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:53.164 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:53.162+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:53.908 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:53.906+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:54.157 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:54.156+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:54.741 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:13:54.953 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:54.951+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:55.072 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:13:55.075 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:13:55.086 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":5780,"stamp":"2023-10-06T00:13:54.088228+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":296,"num_omap_keys":33,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24967,"ondisk_log_size":24967,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001686"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":1926,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050211+0000","last_change":"2023-10-05T21:11:08.661112+0000","last_active":"2023-10-05T22:09:12.050211+0000","last_peered":"2023-10-05T22:09:12.050211+0000","last_clean":"2023-10-05T22:09:12.050211+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-05T22:09:12.050211+0000","last_undegraded":"2023-10-05T22:09:12.050211+0000","last_fullsized":"2023-10-05T22:09:12.050211+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T00:45:46.698848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1829,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050119+0000","last_change":"2023-10-05T21:11:08.660937+0000","last_active":"2023-10-05T22:09:12.050119+0000","last_peered":"2023-10-05T22:09:12.050119+0000","last_clean":"2023-10-05T22:09:12.050119+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-05T22:09:12.050119+0000","last_undegraded":"2023-10-05T22:09:12.050119+0000","last_fullsized":"2023-10-05T22:09:12.050119+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:22.382648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"158'775","reported_seq":1644,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050024+0000","last_change":"2023-10-05T21:11:08.660871+0000","last_active":"2023-10-05T22:09:12.050024+0000","last_peered":"2023-10-05T22:09:12.050024+0000","last_clean":"2023-10-05T22:09:12.050024+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-05T22:09:12.050024+0000","last_undegraded":"2023-10-05T22:09:12.050024+0000","last_fullsized":"2023-10-05T22:09:12.050024+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:17.807335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"158'3217","reported_seq":8586,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050627+0000","last_change":"2023-10-05T21:11:08.651828+0000","last_active":"2023-10-05T22:09:12.050627+0000","last_peered":"2023-10-05T22:09:12.050627+0000","last_clean":"2023-10-05T22:09:12.050627+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-05T22:09:12.050627+0000","last_undegraded":"2023-10-05T22:09:12.050627+0000","last_fullsized":"2023-10-05T22:09:12.050627+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":3217,"log_dups_size":0,"ondisk_log_size":3217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:52:36.783122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11081,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049738+0000","last_change":"2023-10-05T21:11:08.661771+0000","last_active":"2023-10-05T22:09:12.049738+0000","last_peered":"2023-10-05T22:09:12.049738+0000","last_clean":"2023-10-05T22:09:12.049738+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-05T22:09:12.049738+0000","last_undegraded":"2023-10-05T22:09:12.049738+0000","last_fullsized":"2023-10-05T22:09:12.049738+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:30:11.617699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":281,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049561+0000","last_change":"2023-10-05T21:06:29.922199+0000","last_active":"2023-10-05T22:09:12.049561+0000","last_peered":"2023-10-05T22:09:12.049561+0000","last_clean":"2023-10-05T22:09:12.049561+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-05T22:09:12.049561+0000","last_undegraded":"2023-10-05T22:09:12.049561+0000","last_fullsized":"2023-10-05T22:09:12.049561+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:28.614917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:28.614917+0000","last_clean_scrub_stamp":"2023-10-05T21:06:28.614917+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:59:56.310688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2345,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049831+0000","last_change":"2023-10-05T21:11:08.659625+0000","last_active":"2023-10-05T22:09:12.049831+0000","last_peered":"2023-10-05T22:09:12.049831+0000","last_clean":"2023-10-05T22:09:12.049831+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-05T22:09:12.049831+0000","last_undegraded":"2023-10-05T22:09:12.049831+0000","last_fullsized":"2023-10-05T22:09:12.049831+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-06T22:30:43.667801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11729,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049612+0000","last_change":"2023-10-05T21:11:08.661262+0000","last_active":"2023-10-05T22:09:12.049612+0000","last_peered":"2023-10-05T22:09:12.049612+0000","last_clean":"2023-10-05T22:09:12.049612+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-05T22:09:12.049612+0000","last_undegraded":"2023-10-05T22:09:12.049612+0000","last_fullsized":"2023-10-05T22:09:12.049612+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-06T21:11:58.444833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11611,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050624+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-05T22:09:12.050624+0000","last_peered":"2023-10-05T22:09:12.050624+0000","last_clean":"2023-10-05T22:09:12.050624+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-05T22:09:12.050624+0000","last_undegraded":"2023-10-05T22:09:12.050624+0000","last_fullsized":"2023-10-05T22:09:12.050624+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-06T22:30:11.666110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1781,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050624+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-05T22:09:12.050624+0000","last_peered":"2023-10-05T22:09:12.050624+0000","last_clean":"2023-10-05T22:09:12.050624+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-05T22:09:12.050624+0000","last_undegraded":"2023-10-05T22:09:12.050624+0000","last_fullsized":"2023-10-05T22:09:12.050624+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T01:49:48.848559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1750,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049927+0000","last_change":"2023-10-05T21:11:08.661368+0000","last_active":"2023-10-05T22:09:12.049927+0000","last_peered":"2023-10-05T22:09:12.049927+0000","last_clean":"2023-10-05T22:09:12.049927+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-05T22:09:12.049927+0000","last_undegraded":"2023-10-05T22:09:12.049927+0000","last_fullsized":"2023-10-05T22:09:12.049927+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T01:16:04.809466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17193,"ondisk_log_size":17193,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":296,"num_omap_keys":33,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7677,"ondisk_log_size":7677,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740626,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.21199999999999999,"5min":0.222,"15min":0.218},"min":{"1min":0.16600000000000001,"5min":0.16200000000000001,"15min":0.14399999999999999},"max":{"1min":0.25800000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.222},{"interface":"front","average":{"1min":0.23100000000000001,"5min":0.247,"15min":0.245},"min":{"1min":0.187,"5min":0.18099999999999999,"15min":0.16700000000000001},"max":{"1min":0.29799999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.23799999999999999}]},{"osd":1,"last update":"Fri Oct 6 00:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.249,"5min":0.251,"15min":0.24299999999999999},"min":{"1min":0.17299999999999999,"5min":0.17299999999999999,"15min":0.153},"max":{"1min":0.30599999999999999,"5min":0.46700000000000003,"15min":0.48699999999999999},"last":0.253},{"interface":"front","average":{"1min":0.26400000000000001,"5min":0.27500000000000002,"15min":0.27100000000000002},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.16300000000000001},"max":{"1min":0.313,"5min":0.48399999999999999,"15min":0.51200000000000001},"last":0.27300000000000002}]}]},{"osd":0,"up_from":8,"seq":34359740628,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.183,"5min":0.188,"15min":0.20200000000000001},"min":{"1min":0.14299999999999999,"5min":0.128,"15min":0.128},"max":{"1min":0.23799999999999999,"5min":0.34799999999999998,"15min":0.439},"last":0.223},{"interface":"front","average":{"1min":0.26400000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"min":{"1min":0.20699999999999999,"5min":0.193,"15min":0.152},"max":{"1min":0.34899999999999998,"5min":0.40300000000000002,"15min":0.47899999999999998},"last":0.26300000000000001}]},{"osd":2,"last update":"Fri Oct 6 00:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.24199999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"min":{"1min":0.17499999999999999,"5min":0.16300000000000001,"15min":0.16300000000000001},"max":{"1min":0.32100000000000001,"5min":0.36599999999999999,"15min":0.439},"last":0.184},{"interface":"front","average":{"1min":0.29999999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"min":{"1min":0.23300000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.378,"5min":0.436,"15min":0.53600000000000003},"last":0.23100000000000001}]}]},{"osd":1,"up_from":8,"seq":34359740623,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.21299999999999999,"5min":0.20100000000000001,"15min":0.20499999999999999},"min":{"1min":0.13100000000000001,"5min":0.13100000000000001,"15min":0.13},"max":{"1min":0.38400000000000001,"5min":0.38700000000000001,"15min":0.40999999999999998},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.27600000000000002,"5min":0.26800000000000002,"15min":0.27600000000000002},"min":{"1min":0.17599999999999999,"5min":0.16500000000000001,"15min":0.161},"max":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.51000000000000001},"last":0.27500000000000002}]},{"osd":2,"last update":"Fri Oct 6 00:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.246,"5min":0.22900000000000001,"15min":0.23699999999999999},"min":{"1min":0.20200000000000001,"5min":0.17499999999999999,"15min":0.17199999999999999},"max":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.51500000000000001},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.28799999999999998,"5min":0.27000000000000002,"15min":0.27600000000000002},"min":{"1min":0.22600000000000001,"5min":0.20100000000000001,"15min":0.19900000000000001},"max":{"1min":0.439,"5min":0.439,"15min":0.48099999999999998},"last":0.32200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:13:55.088 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:13:55.088 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:13:55.088 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:13:55.088 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:13:55.089 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:13:55.089 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-05T21:06:28.614917+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=28, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:13:55.089 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:13:55.089 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:13:55.089 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:13:55.089 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:13:55.090 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:13:55.090 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:13:55.142 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:55.140+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:55.931 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:55.929+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:56.140 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:56.138+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:56.939 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:56.938+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:57.113 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:57.112+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:57.915 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:57.914+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:58.112 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:58.110+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:58.949 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:58.947+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:59.063 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:13:59.062+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:13:59.955 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:13:59.954+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:00.081 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:00.080+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:01.002 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:01.001+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:01.092 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:01.091+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:02.030 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:02.028+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:02.078 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:02.076+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:03.061 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:03.060+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:03.121 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:03.119+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:04.031 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:04.030+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:04.139 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:04.138+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:05.040 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:05.038+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:05.093 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:05.091+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:05.993 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:05.991+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:06.079 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:06.077+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:07.023 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:07.021+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:07.062 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:07.061+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:08.069 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:08.068+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:08.080 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:08.080+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:09.102 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:09.100+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:09.104 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:09.103+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:10.073 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:10.071+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:10.082 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:10.082+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:11.122 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:11.122+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:11.123 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:11.123+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:12.119 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:12.118+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:12.138 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:12.138+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:13.137 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:13.136+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:13.166 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:13.166+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:14.187 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:14.186+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:14.210 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:14.210+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:15.090 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:14:15.144 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:15.144+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:15.172 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:15.172+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:15.461 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:14:15.463 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:14:15.474 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":5790,"stamp":"2023-10-06T00:14:14.091209+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":296,"num_omap_keys":33,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24967,"ondisk_log_size":24967,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001775"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":1926,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050211+0000","last_change":"2023-10-05T21:11:08.661112+0000","last_active":"2023-10-05T22:09:12.050211+0000","last_peered":"2023-10-05T22:09:12.050211+0000","last_clean":"2023-10-05T22:09:12.050211+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-05T22:09:12.050211+0000","last_undegraded":"2023-10-05T22:09:12.050211+0000","last_fullsized":"2023-10-05T22:09:12.050211+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T00:45:46.698848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1829,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050119+0000","last_change":"2023-10-05T21:11:08.660937+0000","last_active":"2023-10-05T22:09:12.050119+0000","last_peered":"2023-10-05T22:09:12.050119+0000","last_clean":"2023-10-05T22:09:12.050119+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-05T22:09:12.050119+0000","last_undegraded":"2023-10-05T22:09:12.050119+0000","last_fullsized":"2023-10-05T22:09:12.050119+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:22.382648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"158'775","reported_seq":1644,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050024+0000","last_change":"2023-10-05T21:11:08.660871+0000","last_active":"2023-10-05T22:09:12.050024+0000","last_peered":"2023-10-05T22:09:12.050024+0000","last_clean":"2023-10-05T22:09:12.050024+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-05T22:09:12.050024+0000","last_undegraded":"2023-10-05T22:09:12.050024+0000","last_fullsized":"2023-10-05T22:09:12.050024+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:17.807335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"158'3217","reported_seq":8586,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050627+0000","last_change":"2023-10-05T21:11:08.651828+0000","last_active":"2023-10-05T22:09:12.050627+0000","last_peered":"2023-10-05T22:09:12.050627+0000","last_clean":"2023-10-05T22:09:12.050627+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-05T22:09:12.050627+0000","last_undegraded":"2023-10-05T22:09:12.050627+0000","last_fullsized":"2023-10-05T22:09:12.050627+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":3217,"log_dups_size":0,"ondisk_log_size":3217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:52:36.783122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11081,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049738+0000","last_change":"2023-10-05T21:11:08.661771+0000","last_active":"2023-10-05T22:09:12.049738+0000","last_peered":"2023-10-05T22:09:12.049738+0000","last_clean":"2023-10-05T22:09:12.049738+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-05T22:09:12.049738+0000","last_undegraded":"2023-10-05T22:09:12.049738+0000","last_fullsized":"2023-10-05T22:09:12.049738+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:30:11.617699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":281,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049561+0000","last_change":"2023-10-05T21:06:29.922199+0000","last_active":"2023-10-05T22:09:12.049561+0000","last_peered":"2023-10-05T22:09:12.049561+0000","last_clean":"2023-10-05T22:09:12.049561+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-05T22:09:12.049561+0000","last_undegraded":"2023-10-05T22:09:12.049561+0000","last_fullsized":"2023-10-05T22:09:12.049561+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:28.614917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:28.614917+0000","last_clean_scrub_stamp":"2023-10-05T21:06:28.614917+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:59:56.310688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2345,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049831+0000","last_change":"2023-10-05T21:11:08.659625+0000","last_active":"2023-10-05T22:09:12.049831+0000","last_peered":"2023-10-05T22:09:12.049831+0000","last_clean":"2023-10-05T22:09:12.049831+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-05T22:09:12.049831+0000","last_undegraded":"2023-10-05T22:09:12.049831+0000","last_fullsized":"2023-10-05T22:09:12.049831+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-06T22:30:43.667801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11729,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049612+0000","last_change":"2023-10-05T21:11:08.661262+0000","last_active":"2023-10-05T22:09:12.049612+0000","last_peered":"2023-10-05T22:09:12.049612+0000","last_clean":"2023-10-05T22:09:12.049612+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-05T22:09:12.049612+0000","last_undegraded":"2023-10-05T22:09:12.049612+0000","last_fullsized":"2023-10-05T22:09:12.049612+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-06T21:11:58.444833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11611,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050624+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-05T22:09:12.050624+0000","last_peered":"2023-10-05T22:09:12.050624+0000","last_clean":"2023-10-05T22:09:12.050624+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-05T22:09:12.050624+0000","last_undegraded":"2023-10-05T22:09:12.050624+0000","last_fullsized":"2023-10-05T22:09:12.050624+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-06T22:30:11.666110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1781,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050624+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-05T22:09:12.050624+0000","last_peered":"2023-10-05T22:09:12.050624+0000","last_clean":"2023-10-05T22:09:12.050624+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-05T22:09:12.050624+0000","last_undegraded":"2023-10-05T22:09:12.050624+0000","last_fullsized":"2023-10-05T22:09:12.050624+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T01:49:48.848559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1750,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049927+0000","last_change":"2023-10-05T21:11:08.661368+0000","last_active":"2023-10-05T22:09:12.049927+0000","last_peered":"2023-10-05T22:09:12.049927+0000","last_clean":"2023-10-05T22:09:12.049927+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-05T22:09:12.049927+0000","last_undegraded":"2023-10-05T22:09:12.049927+0000","last_fullsized":"2023-10-05T22:09:12.049927+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T01:16:04.809466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17193,"ondisk_log_size":17193,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":296,"num_omap_keys":33,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7677,"ondisk_log_size":7677,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740630,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:13:57 2023","interfaces":[{"interface":"back","average":{"1min":0.214,"5min":0.219,"15min":0.218},"min":{"1min":0.155,"5min":0.155,"15min":0.14399999999999999},"max":{"1min":0.39100000000000001,"5min":0.45900000000000002,"15min":0.46999999999999997},"last":0.20599999999999999},{"interface":"front","average":{"1min":0.23699999999999999,"5min":0.24299999999999999,"15min":0.24399999999999999},"min":{"1min":0.17599999999999999,"5min":0.17599999999999999,"15min":0.16700000000000001},"max":{"1min":0.378,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.23400000000000001}]},{"osd":1,"last update":"Fri Oct 6 00:13:57 2023","interfaces":[{"interface":"back","average":{"1min":0.247,"5min":0.25,"15min":0.24199999999999999},"min":{"1min":0.17699999999999999,"5min":0.17299999999999999,"15min":0.153},"max":{"1min":0.41499999999999998,"5min":0.46700000000000003,"15min":0.48699999999999999},"last":0.221},{"interface":"front","average":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27000000000000002},"min":{"1min":0.217,"5min":0.19600000000000001,"15min":0.16300000000000001},"max":{"1min":0.41199999999999998,"5min":0.45100000000000001,"15min":0.51200000000000001},"last":0.26400000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740632,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.216,"5min":0.19,"15min":0.20200000000000001},"min":{"1min":0.154,"5min":0.128,"15min":0.128},"max":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.439},"last":0.215},{"interface":"front","average":{"1min":0.28699999999999998,"5min":0.26500000000000001,"15min":0.26300000000000001},"min":{"1min":0.22500000000000001,"5min":0.193,"15min":0.152},"max":{"1min":0.434,"5min":0.434,"15min":0.47899999999999998},"last":0.28599999999999998}]},{"osd":2,"last update":"Fri Oct 6 00:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.252,"5min":0.23499999999999999,"15min":0.23200000000000001},"min":{"1min":0.184,"5min":0.16300000000000001,"15min":0.16300000000000001},"max":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.439},"last":0.252},{"interface":"front","average":{"1min":0.316,"5min":0.29799999999999999,"15min":0.29599999999999999},"min":{"1min":0.23100000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.53600000000000003},"last":0.309}]}]},{"osd":1,"up_from":8,"seq":34359740627,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.21299999999999999,"5min":0.20100000000000001,"15min":0.20499999999999999},"min":{"1min":0.13100000000000001,"5min":0.13100000000000001,"15min":0.13},"max":{"1min":0.38400000000000001,"5min":0.38700000000000001,"15min":0.40999999999999998},"last":0.16400000000000001},{"interface":"front","average":{"1min":0.27600000000000002,"5min":0.26800000000000002,"15min":0.27600000000000002},"min":{"1min":0.17599999999999999,"5min":0.16500000000000001,"15min":0.161},"max":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.51000000000000001},"last":0.23100000000000001}]},{"osd":2,"last update":"Fri Oct 6 00:13:26 2023","interfaces":[{"interface":"back","average":{"1min":0.246,"5min":0.22900000000000001,"15min":0.23699999999999999},"min":{"1min":0.20200000000000001,"5min":0.17499999999999999,"15min":0.17199999999999999},"max":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.51500000000000001},"last":0.20200000000000001},{"interface":"front","average":{"1min":0.28799999999999998,"5min":0.27000000000000002,"15min":0.27600000000000002},"min":{"1min":0.22600000000000001,"5min":0.20100000000000001,"15min":0.19900000000000001},"max":{"1min":0.439,"5min":0.439,"15min":0.48099999999999998},"last":0.252}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:14:15.475 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:14:15.476 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:14:15.476 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:14:15.476 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:14:15.476 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:14:15.476 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-05T21:06:28.614917+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=28, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:14:15.477 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:14:15.477 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:14:15.477 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:14:15.477 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:14:15.477 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:14:15.477 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:14:16.097 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:16.096+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:16.174 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:16.173+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:17.129 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:17.128+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:17.148 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:17.148+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:18.116 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:18.116+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:18.164 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:18.164+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:19.097 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:19.096+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:19.191 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:19.190+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:20.068 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:20.067+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:20.203 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:20.203+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:21.038 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:21.037+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:21.205 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:21.204+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:22.039 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:22.038+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:22.203 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:22.202+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:23.036 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:23.036+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:23.242 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:23.242+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:24.031 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:24.030+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:24.208 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:24.207+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:24.989 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:24.989+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:25.218 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:25.218+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:25.986 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:25.986+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:26.257 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:26.256+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:26.940 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:26.940+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:27.265 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:27.265+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:27.923 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:27.923+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:28.226 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:28.225+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:28.893 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:28.892+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:29.241 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:29.240+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:29.934 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:29.933+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:30.235 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:30.233+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:30.939 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:30.937+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:31.284 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:31.282+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:31.980 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:31.979+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:32.251 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:32.250+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:33.006 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:33.004+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:33.235 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:33.234+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:34.014 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:34.013+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:34.260 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:34.258+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:35.062 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:35.061+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:35.241 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:35.240+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:35.479 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:14:35.823 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:14:35.826 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:14:35.837 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":5800,"stamp":"2023-10-06T00:14:34.094343+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":296,"num_omap_keys":33,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24967,"ondisk_log_size":24967,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001975"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":1926,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050211+0000","last_change":"2023-10-05T21:11:08.661112+0000","last_active":"2023-10-05T22:09:12.050211+0000","last_peered":"2023-10-05T22:09:12.050211+0000","last_clean":"2023-10-05T22:09:12.050211+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-05T22:09:12.050211+0000","last_undegraded":"2023-10-05T22:09:12.050211+0000","last_fullsized":"2023-10-05T22:09:12.050211+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T00:45:46.698848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1829,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050119+0000","last_change":"2023-10-05T21:11:08.660937+0000","last_active":"2023-10-05T22:09:12.050119+0000","last_peered":"2023-10-05T22:09:12.050119+0000","last_clean":"2023-10-05T22:09:12.050119+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-05T22:09:12.050119+0000","last_undegraded":"2023-10-05T22:09:12.050119+0000","last_fullsized":"2023-10-05T22:09:12.050119+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:22.382648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"158'775","reported_seq":1644,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050024+0000","last_change":"2023-10-05T21:11:08.660871+0000","last_active":"2023-10-05T22:09:12.050024+0000","last_peered":"2023-10-05T22:09:12.050024+0000","last_clean":"2023-10-05T22:09:12.050024+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-05T22:09:12.050024+0000","last_undegraded":"2023-10-05T22:09:12.050024+0000","last_fullsized":"2023-10-05T22:09:12.050024+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:17.807335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"158'3217","reported_seq":8586,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050627+0000","last_change":"2023-10-05T21:11:08.651828+0000","last_active":"2023-10-05T22:09:12.050627+0000","last_peered":"2023-10-05T22:09:12.050627+0000","last_clean":"2023-10-05T22:09:12.050627+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-05T22:09:12.050627+0000","last_undegraded":"2023-10-05T22:09:12.050627+0000","last_fullsized":"2023-10-05T22:09:12.050627+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":3217,"log_dups_size":0,"ondisk_log_size":3217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:52:36.783122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11081,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049738+0000","last_change":"2023-10-05T21:11:08.661771+0000","last_active":"2023-10-05T22:09:12.049738+0000","last_peered":"2023-10-05T22:09:12.049738+0000","last_clean":"2023-10-05T22:09:12.049738+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-05T22:09:12.049738+0000","last_undegraded":"2023-10-05T22:09:12.049738+0000","last_fullsized":"2023-10-05T22:09:12.049738+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:30:11.617699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":281,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049561+0000","last_change":"2023-10-05T21:06:29.922199+0000","last_active":"2023-10-05T22:09:12.049561+0000","last_peered":"2023-10-05T22:09:12.049561+0000","last_clean":"2023-10-05T22:09:12.049561+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-05T22:09:12.049561+0000","last_undegraded":"2023-10-05T22:09:12.049561+0000","last_fullsized":"2023-10-05T22:09:12.049561+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:28.614917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:28.614917+0000","last_clean_scrub_stamp":"2023-10-05T21:06:28.614917+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:59:56.310688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2345,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049831+0000","last_change":"2023-10-05T21:11:08.659625+0000","last_active":"2023-10-05T22:09:12.049831+0000","last_peered":"2023-10-05T22:09:12.049831+0000","last_clean":"2023-10-05T22:09:12.049831+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-05T22:09:12.049831+0000","last_undegraded":"2023-10-05T22:09:12.049831+0000","last_fullsized":"2023-10-05T22:09:12.049831+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-06T22:30:43.667801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11729,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049612+0000","last_change":"2023-10-05T21:11:08.661262+0000","last_active":"2023-10-05T22:09:12.049612+0000","last_peered":"2023-10-05T22:09:12.049612+0000","last_clean":"2023-10-05T22:09:12.049612+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-05T22:09:12.049612+0000","last_undegraded":"2023-10-05T22:09:12.049612+0000","last_fullsized":"2023-10-05T22:09:12.049612+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-06T21:11:58.444833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11611,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050624+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-05T22:09:12.050624+0000","last_peered":"2023-10-05T22:09:12.050624+0000","last_clean":"2023-10-05T22:09:12.050624+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-05T22:09:12.050624+0000","last_undegraded":"2023-10-05T22:09:12.050624+0000","last_fullsized":"2023-10-05T22:09:12.050624+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-06T22:30:11.666110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1781,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050624+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-05T22:09:12.050624+0000","last_peered":"2023-10-05T22:09:12.050624+0000","last_clean":"2023-10-05T22:09:12.050624+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-05T22:09:12.050624+0000","last_undegraded":"2023-10-05T22:09:12.050624+0000","last_fullsized":"2023-10-05T22:09:12.050624+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T01:49:48.848559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1750,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049927+0000","last_change":"2023-10-05T21:11:08.661368+0000","last_active":"2023-10-05T22:09:12.049927+0000","last_peered":"2023-10-05T22:09:12.049927+0000","last_clean":"2023-10-05T22:09:12.049927+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-05T22:09:12.049927+0000","last_undegraded":"2023-10-05T22:09:12.049927+0000","last_fullsized":"2023-10-05T22:09:12.049927+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T01:16:04.809466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17193,"ondisk_log_size":17193,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":296,"num_omap_keys":33,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7677,"ondisk_log_size":7677,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740634,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:13:57 2023","interfaces":[{"interface":"back","average":{"1min":0.214,"5min":0.219,"15min":0.218},"min":{"1min":0.155,"5min":0.155,"15min":0.14399999999999999},"max":{"1min":0.39100000000000001,"5min":0.45900000000000002,"15min":0.46999999999999997},"last":0.215},{"interface":"front","average":{"1min":0.23699999999999999,"5min":0.24299999999999999,"15min":0.24399999999999999},"min":{"1min":0.17599999999999999,"5min":0.17599999999999999,"15min":0.16700000000000001},"max":{"1min":0.378,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.22800000000000001}]},{"osd":1,"last update":"Fri Oct 6 00:13:57 2023","interfaces":[{"interface":"back","average":{"1min":0.247,"5min":0.25,"15min":0.24199999999999999},"min":{"1min":0.17699999999999999,"5min":0.17299999999999999,"15min":0.153},"max":{"1min":0.41499999999999998,"5min":0.46700000000000003,"15min":0.48699999999999999},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27000000000000002},"min":{"1min":0.217,"5min":0.19600000000000001,"15min":0.16300000000000001},"max":{"1min":0.41199999999999998,"5min":0.45100000000000001,"15min":0.51200000000000001},"last":0.32400000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740636,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.216,"5min":0.19,"15min":0.20200000000000001},"min":{"1min":0.154,"5min":0.128,"15min":0.128},"max":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.439},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.28699999999999998,"5min":0.26500000000000001,"15min":0.26300000000000001},"min":{"1min":0.22500000000000001,"5min":0.193,"15min":0.152},"max":{"1min":0.434,"5min":0.434,"15min":0.47899999999999998},"last":0.33200000000000002}]},{"osd":2,"last update":"Fri Oct 6 00:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.252,"5min":0.23499999999999999,"15min":0.23200000000000001},"min":{"1min":0.184,"5min":0.16300000000000001,"15min":0.16300000000000001},"max":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.439},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.316,"5min":0.29799999999999999,"15min":0.29599999999999999},"min":{"1min":0.23100000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.53600000000000003},"last":0.35599999999999998}]}]},{"osd":1,"up_from":8,"seq":34359740631,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.22500000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"min":{"1min":0.14899999999999999,"5min":0.13100000000000001,"15min":0.13},"max":{"1min":0.497,"5min":0.497,"15min":0.497},"last":0.17899999999999999},{"interface":"front","average":{"1min":0.25800000000000001,"5min":0.26600000000000001,"15min":0.27500000000000002},"min":{"1min":0.154,"5min":0.154,"15min":0.154},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.22900000000000001}]},{"osd":2,"last update":"Fri Oct 6 00:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.23499999999999999,"5min":0.23000000000000001,"15min":0.23699999999999999},"min":{"1min":0.17100000000000001,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.51500000000000001},"last":0.20399999999999999},{"interface":"front","average":{"1min":0.27600000000000002,"5min":0.27100000000000002,"15min":0.27600000000000002},"min":{"1min":0.20999999999999999,"5min":0.20100000000000001,"15min":0.19900000000000001},"max":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.249}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:14:35.838 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:14:35.838 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:14:35.839 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:14:35.839 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:14:35.839 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:14:35.839 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-05T21:06:28.614917+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=28, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:14:35.839 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:14:35.840 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:14:35.840 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:14:35.840 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:14:35.840 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:14:35.840 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:14:36.018 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:36.017+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:36.226 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:36.224+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:36.985 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:36.983+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:37.203 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:37.201+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:38.010 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:38.008+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:38.166 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:38.165+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:39.037 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:39.035+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:39.158 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:39.156+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:40.060 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:40.058+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:40.135 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:40.134+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:41.016 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:41.015+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:41.119 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:41.118+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:42.058 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:42.057+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:42.148 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:42.147+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:43.047 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:43.045+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:43.142 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:43.141+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:44.006 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:44.004+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:44.190 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:44.189+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:45.034 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:45.032+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:45.202 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:45.201+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:46.006 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:46.005+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:46.193 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:46.191+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:47.004 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:47.002+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:47.146 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:47.145+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:47.982 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:47.980+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:48.099 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:48.098+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:48.984 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:48.982+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:49.128 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:49.127+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:49.987 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:49.985+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:50.091 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:50.089+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:51.022 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:51.021+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:51.046 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:51.045+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:52.018 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:52.016+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:52.095 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:52.093+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:53.058 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:53.056+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:53.064 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:53.062+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:54.014 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:54.012+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:54.082 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:54.081+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:55.011 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:55.010+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:55.094 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:55.092+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:55.841 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:14:56.033 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:56.031+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:56.060 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:56.058+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:56.175 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:14:56.177 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:14:56.188 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":5811,"stamp":"2023-10-06T00:14:56.097673+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":296,"num_omap_keys":33,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24967,"ondisk_log_size":24967,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001903"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":1926,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050211+0000","last_change":"2023-10-05T21:11:08.661112+0000","last_active":"2023-10-05T22:09:12.050211+0000","last_peered":"2023-10-05T22:09:12.050211+0000","last_clean":"2023-10-05T22:09:12.050211+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-05T22:09:12.050211+0000","last_undegraded":"2023-10-05T22:09:12.050211+0000","last_fullsized":"2023-10-05T22:09:12.050211+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T00:45:46.698848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1829,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050119+0000","last_change":"2023-10-05T21:11:08.660937+0000","last_active":"2023-10-05T22:09:12.050119+0000","last_peered":"2023-10-05T22:09:12.050119+0000","last_clean":"2023-10-05T22:09:12.050119+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-05T22:09:12.050119+0000","last_undegraded":"2023-10-05T22:09:12.050119+0000","last_fullsized":"2023-10-05T22:09:12.050119+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:22.382648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"158'775","reported_seq":1644,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050024+0000","last_change":"2023-10-05T21:11:08.660871+0000","last_active":"2023-10-05T22:09:12.050024+0000","last_peered":"2023-10-05T22:09:12.050024+0000","last_clean":"2023-10-05T22:09:12.050024+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-05T22:09:12.050024+0000","last_undegraded":"2023-10-05T22:09:12.050024+0000","last_fullsized":"2023-10-05T22:09:12.050024+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:17.807335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"158'3217","reported_seq":8586,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050627+0000","last_change":"2023-10-05T21:11:08.651828+0000","last_active":"2023-10-05T22:09:12.050627+0000","last_peered":"2023-10-05T22:09:12.050627+0000","last_clean":"2023-10-05T22:09:12.050627+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-05T22:09:12.050627+0000","last_undegraded":"2023-10-05T22:09:12.050627+0000","last_fullsized":"2023-10-05T22:09:12.050627+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":3217,"log_dups_size":0,"ondisk_log_size":3217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:52:36.783122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11081,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049738+0000","last_change":"2023-10-05T21:11:08.661771+0000","last_active":"2023-10-05T22:09:12.049738+0000","last_peered":"2023-10-05T22:09:12.049738+0000","last_clean":"2023-10-05T22:09:12.049738+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-05T22:09:12.049738+0000","last_undegraded":"2023-10-05T22:09:12.049738+0000","last_fullsized":"2023-10-05T22:09:12.049738+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:30:11.617699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":281,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049561+0000","last_change":"2023-10-05T21:06:29.922199+0000","last_active":"2023-10-05T22:09:12.049561+0000","last_peered":"2023-10-05T22:09:12.049561+0000","last_clean":"2023-10-05T22:09:12.049561+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-05T22:09:12.049561+0000","last_undegraded":"2023-10-05T22:09:12.049561+0000","last_fullsized":"2023-10-05T22:09:12.049561+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:28.614917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:28.614917+0000","last_clean_scrub_stamp":"2023-10-05T21:06:28.614917+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:59:56.310688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2345,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049831+0000","last_change":"2023-10-05T21:11:08.659625+0000","last_active":"2023-10-05T22:09:12.049831+0000","last_peered":"2023-10-05T22:09:12.049831+0000","last_clean":"2023-10-05T22:09:12.049831+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-05T22:09:12.049831+0000","last_undegraded":"2023-10-05T22:09:12.049831+0000","last_fullsized":"2023-10-05T22:09:12.049831+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-06T22:30:43.667801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11729,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049612+0000","last_change":"2023-10-05T21:11:08.661262+0000","last_active":"2023-10-05T22:09:12.049612+0000","last_peered":"2023-10-05T22:09:12.049612+0000","last_clean":"2023-10-05T22:09:12.049612+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-05T22:09:12.049612+0000","last_undegraded":"2023-10-05T22:09:12.049612+0000","last_fullsized":"2023-10-05T22:09:12.049612+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-06T21:11:58.444833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11611,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050624+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-05T22:09:12.050624+0000","last_peered":"2023-10-05T22:09:12.050624+0000","last_clean":"2023-10-05T22:09:12.050624+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-05T22:09:12.050624+0000","last_undegraded":"2023-10-05T22:09:12.050624+0000","last_fullsized":"2023-10-05T22:09:12.050624+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-06T22:30:11.666110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1781,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050624+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-05T22:09:12.050624+0000","last_peered":"2023-10-05T22:09:12.050624+0000","last_clean":"2023-10-05T22:09:12.050624+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-05T22:09:12.050624+0000","last_undegraded":"2023-10-05T22:09:12.050624+0000","last_fullsized":"2023-10-05T22:09:12.050624+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T01:49:48.848559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1750,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049927+0000","last_change":"2023-10-05T21:11:08.661368+0000","last_active":"2023-10-05T22:09:12.049927+0000","last_peered":"2023-10-05T22:09:12.049927+0000","last_clean":"2023-10-05T22:09:12.049927+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-05T22:09:12.049927+0000","last_undegraded":"2023-10-05T22:09:12.049927+0000","last_fullsized":"2023-10-05T22:09:12.049927+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T01:16:04.809466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17193,"ondisk_log_size":17193,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":296,"num_omap_keys":33,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7677,"ondisk_log_size":7677,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740639,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:13:57 2023","interfaces":[{"interface":"back","average":{"1min":0.214,"5min":0.219,"15min":0.218},"min":{"1min":0.155,"5min":0.155,"15min":0.14399999999999999},"max":{"1min":0.39100000000000001,"5min":0.45900000000000002,"15min":0.46999999999999997},"last":0.219},{"interface":"front","average":{"1min":0.23699999999999999,"5min":0.24299999999999999,"15min":0.24399999999999999},"min":{"1min":0.17599999999999999,"5min":0.17599999999999999,"15min":0.16700000000000001},"max":{"1min":0.378,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.20799999999999999}]},{"osd":1,"last update":"Fri Oct 6 00:13:57 2023","interfaces":[{"interface":"back","average":{"1min":0.247,"5min":0.25,"15min":0.24199999999999999},"min":{"1min":0.17699999999999999,"5min":0.17299999999999999,"15min":0.153},"max":{"1min":0.41499999999999998,"5min":0.46700000000000003,"15min":0.48699999999999999},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27000000000000002},"min":{"1min":0.217,"5min":0.19600000000000001,"15min":0.16300000000000001},"max":{"1min":0.41199999999999998,"5min":0.45100000000000001,"15min":0.51200000000000001},"last":0.27700000000000002}]}]},{"osd":0,"up_from":8,"seq":34359740640,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.216,"5min":0.19,"15min":0.20200000000000001},"min":{"1min":0.154,"5min":0.128,"15min":0.128},"max":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.439},"last":0.17799999999999999},{"interface":"front","average":{"1min":0.28699999999999998,"5min":0.26500000000000001,"15min":0.26300000000000001},"min":{"1min":0.22500000000000001,"5min":0.193,"15min":0.152},"max":{"1min":0.434,"5min":0.434,"15min":0.47899999999999998},"last":0.24199999999999999}]},{"osd":2,"last update":"Fri Oct 6 00:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.252,"5min":0.23499999999999999,"15min":0.23200000000000001},"min":{"1min":0.184,"5min":0.16300000000000001,"15min":0.16300000000000001},"max":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.439},"last":0.20899999999999999},{"interface":"front","average":{"1min":0.316,"5min":0.29799999999999999,"15min":0.29599999999999999},"min":{"1min":0.23100000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.53600000000000003},"last":0.28299999999999997}]}]},{"osd":1,"up_from":8,"seq":34359740635,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.22500000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"min":{"1min":0.14899999999999999,"5min":0.13100000000000001,"15min":0.13},"max":{"1min":0.497,"5min":0.497,"15min":0.497},"last":0.255},{"interface":"front","average":{"1min":0.25800000000000001,"5min":0.26600000000000001,"15min":0.27500000000000002},"min":{"1min":0.154,"5min":0.154,"15min":0.154},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.28899999999999998}]},{"osd":2,"last update":"Fri Oct 6 00:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.23499999999999999,"5min":0.23000000000000001,"15min":0.23699999999999999},"min":{"1min":0.17100000000000001,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.51500000000000001},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.27600000000000002,"5min":0.27100000000000002,"15min":0.27600000000000002},"min":{"1min":0.20999999999999999,"5min":0.20100000000000001,"15min":0.19900000000000001},"max":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.26400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:14:56.190 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:14:56.190 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:14:56.190 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:14:56.191 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:14:56.191 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:14:56.191 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-05T21:06:28.614917+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=28, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:14:56.191 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:14:56.191 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:14:56.192 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:14:56.192 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:14:56.192 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:14:56.192 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:14:57.078 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:57.077+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:57.088 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:57.087+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:58.062 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:58.060+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:58.124 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:58.122+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:59.037 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:14:59.035+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:14:59.155 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:14:59.153+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:00.012 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:00.011+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:00.145 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:00.144+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:01.058 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:01.056+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:01.193 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:01.191+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:02.064 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:02.062+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:02.210 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:02.208+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:03.080 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:03.079+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:03.249 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:03.248+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:04.038 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:04.036+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:04.240 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:04.239+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:04.999 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:04.998+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:05.242 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:05.241+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:06.048 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:06.047+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:06.253 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:06.252+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:07.060 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:07.058+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:07.277 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:07.275+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:08.012 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:08.011+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:08.253 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:08.253+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:08.994 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:08.994+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:09.271 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:09.271+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:09.986 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:09.985+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:10.298 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:10.297+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:10.994 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:10.994+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:11.286 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:11.286+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:12.023 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:12.023+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:12.287 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:12.287+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:12.990 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:12.989+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:13.254 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:13.254+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:13.947 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:13.946+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:14.240 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:14.239+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:14.973 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:14.972+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:15.263 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:15.262+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:15.988 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:15.987+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:16.193 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:15:16.252 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:16.252+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:16.558 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:15:16.560 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:15:16.571 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":5821,"stamp":"2023-10-06T00:15:16.100938+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":296,"num_omap_keys":33,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24967,"ondisk_log_size":24967,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002028"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":1926,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050211+0000","last_change":"2023-10-05T21:11:08.661112+0000","last_active":"2023-10-05T22:09:12.050211+0000","last_peered":"2023-10-05T22:09:12.050211+0000","last_clean":"2023-10-05T22:09:12.050211+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-05T22:09:12.050211+0000","last_undegraded":"2023-10-05T22:09:12.050211+0000","last_fullsized":"2023-10-05T22:09:12.050211+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T00:45:46.698848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1829,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050119+0000","last_change":"2023-10-05T21:11:08.660937+0000","last_active":"2023-10-05T22:09:12.050119+0000","last_peered":"2023-10-05T22:09:12.050119+0000","last_clean":"2023-10-05T22:09:12.050119+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-05T22:09:12.050119+0000","last_undegraded":"2023-10-05T22:09:12.050119+0000","last_fullsized":"2023-10-05T22:09:12.050119+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:22.382648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"158'775","reported_seq":1644,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050024+0000","last_change":"2023-10-05T21:11:08.660871+0000","last_active":"2023-10-05T22:09:12.050024+0000","last_peered":"2023-10-05T22:09:12.050024+0000","last_clean":"2023-10-05T22:09:12.050024+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-05T22:09:12.050024+0000","last_undegraded":"2023-10-05T22:09:12.050024+0000","last_fullsized":"2023-10-05T22:09:12.050024+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:17.807335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"158'3217","reported_seq":8586,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050627+0000","last_change":"2023-10-05T21:11:08.651828+0000","last_active":"2023-10-05T22:09:12.050627+0000","last_peered":"2023-10-05T22:09:12.050627+0000","last_clean":"2023-10-05T22:09:12.050627+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-05T22:09:12.050627+0000","last_undegraded":"2023-10-05T22:09:12.050627+0000","last_fullsized":"2023-10-05T22:09:12.050627+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":3217,"log_dups_size":0,"ondisk_log_size":3217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:52:36.783122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11081,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049738+0000","last_change":"2023-10-05T21:11:08.661771+0000","last_active":"2023-10-05T22:09:12.049738+0000","last_peered":"2023-10-05T22:09:12.049738+0000","last_clean":"2023-10-05T22:09:12.049738+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-05T22:09:12.049738+0000","last_undegraded":"2023-10-05T22:09:12.049738+0000","last_fullsized":"2023-10-05T22:09:12.049738+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:30:11.617699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":281,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049561+0000","last_change":"2023-10-05T21:06:29.922199+0000","last_active":"2023-10-05T22:09:12.049561+0000","last_peered":"2023-10-05T22:09:12.049561+0000","last_clean":"2023-10-05T22:09:12.049561+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-05T22:09:12.049561+0000","last_undegraded":"2023-10-05T22:09:12.049561+0000","last_fullsized":"2023-10-05T22:09:12.049561+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:28.614917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:28.614917+0000","last_clean_scrub_stamp":"2023-10-05T21:06:28.614917+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:59:56.310688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2345,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049831+0000","last_change":"2023-10-05T21:11:08.659625+0000","last_active":"2023-10-05T22:09:12.049831+0000","last_peered":"2023-10-05T22:09:12.049831+0000","last_clean":"2023-10-05T22:09:12.049831+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-05T22:09:12.049831+0000","last_undegraded":"2023-10-05T22:09:12.049831+0000","last_fullsized":"2023-10-05T22:09:12.049831+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-06T22:30:43.667801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11729,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049612+0000","last_change":"2023-10-05T21:11:08.661262+0000","last_active":"2023-10-05T22:09:12.049612+0000","last_peered":"2023-10-05T22:09:12.049612+0000","last_clean":"2023-10-05T22:09:12.049612+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-05T22:09:12.049612+0000","last_undegraded":"2023-10-05T22:09:12.049612+0000","last_fullsized":"2023-10-05T22:09:12.049612+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-06T21:11:58.444833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11611,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050624+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-05T22:09:12.050624+0000","last_peered":"2023-10-05T22:09:12.050624+0000","last_clean":"2023-10-05T22:09:12.050624+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-05T22:09:12.050624+0000","last_undegraded":"2023-10-05T22:09:12.050624+0000","last_fullsized":"2023-10-05T22:09:12.050624+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-06T22:30:11.666110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1781,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050624+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-05T22:09:12.050624+0000","last_peered":"2023-10-05T22:09:12.050624+0000","last_clean":"2023-10-05T22:09:12.050624+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-05T22:09:12.050624+0000","last_undegraded":"2023-10-05T22:09:12.050624+0000","last_fullsized":"2023-10-05T22:09:12.050624+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T01:49:48.848559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1750,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049927+0000","last_change":"2023-10-05T21:11:08.661368+0000","last_active":"2023-10-05T22:09:12.049927+0000","last_peered":"2023-10-05T22:09:12.049927+0000","last_clean":"2023-10-05T22:09:12.049927+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-05T22:09:12.049927+0000","last_undegraded":"2023-10-05T22:09:12.049927+0000","last_fullsized":"2023-10-05T22:09:12.049927+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T01:16:04.809466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17193,"ondisk_log_size":17193,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":296,"num_omap_keys":33,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7677,"ondisk_log_size":7677,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740643,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:14:59 2023","interfaces":[{"interface":"back","average":{"1min":0.21199999999999999,"5min":0.217,"15min":0.217},"min":{"1min":0.16400000000000001,"5min":0.155,"15min":0.14399999999999999},"max":{"1min":0.32100000000000001,"5min":0.45900000000000002,"15min":0.46999999999999997},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.23100000000000001,"5min":0.23899999999999999,"15min":0.24299999999999999},"min":{"1min":0.17799999999999999,"5min":0.17599999999999999,"15min":0.16700000000000001},"max":{"1min":0.41899999999999998,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.35399999999999998}]},{"osd":1,"last update":"Fri Oct 6 00:14:59 2023","interfaces":[{"interface":"back","average":{"1min":0.248,"5min":0.249,"15min":0.24199999999999999},"min":{"1min":0.191,"5min":0.17299999999999999,"15min":0.153},"max":{"1min":0.32100000000000001,"5min":0.45400000000000001,"15min":0.48699999999999999},"last":0.37},{"interface":"front","average":{"1min":0.26400000000000001,"5min":0.27000000000000002,"15min":0.26900000000000002},"min":{"1min":0.223,"5min":0.19600000000000001,"15min":0.16300000000000001},"max":{"1min":0.32400000000000001,"5min":0.442,"15min":0.51200000000000001},"last":0.39100000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740644,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.19500000000000001,"5min":0.191,"15min":0.20000000000000001},"min":{"1min":0.13600000000000001,"5min":0.128,"15min":0.128},"max":{"1min":0.309,"5min":0.34999999999999998,"15min":0.439},"last":0.17299999999999999},{"interface":"front","average":{"1min":0.26800000000000002,"5min":0.26600000000000001,"15min":0.26200000000000001},"min":{"1min":0.189,"5min":0.189,"15min":0.152},"max":{"1min":0.33200000000000002,"5min":0.434,"15min":0.47899999999999998},"last":0.25600000000000001}]},{"osd":2,"last update":"Fri Oct 6 00:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.22800000000000001,"5min":0.23400000000000001,"15min":0.23100000000000001},"min":{"1min":0.16700000000000001,"5min":0.16300000000000001,"15min":0.16300000000000001},"max":{"1min":0.30399999999999999,"5min":0.36799999999999999,"15min":0.439},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.28699999999999998,"5min":0.29599999999999999,"15min":0.29399999999999998},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.35599999999999998,"5min":0.46400000000000002,"15min":0.53600000000000003},"last":0.28000000000000003}]}]},{"osd":1,"up_from":8,"seq":34359740639,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.22500000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"min":{"1min":0.14899999999999999,"5min":0.13100000000000001,"15min":0.13},"max":{"1min":0.497,"5min":0.497,"15min":0.497},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.25800000000000001,"5min":0.26600000000000001,"15min":0.27500000000000002},"min":{"1min":0.154,"5min":0.154,"15min":0.154},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.26000000000000001}]},{"osd":2,"last update":"Fri Oct 6 00:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.23499999999999999,"5min":0.23000000000000001,"15min":0.23699999999999999},"min":{"1min":0.17100000000000001,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.51500000000000001},"last":0.224},{"interface":"front","average":{"1min":0.27600000000000002,"5min":0.27100000000000002,"15min":0.27600000000000002},"min":{"1min":0.20999999999999999,"5min":0.20100000000000001,"15min":0.19900000000000001},"max":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.29199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:15:16.573 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:15:16.573 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:15:16.573 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:15:16.574 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:15:16.574 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:15:16.574 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-05T21:06:28.614917+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=28, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:15:16.574 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:15:16.574 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:15:16.575 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:15:16.575 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:15:16.575 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:15:16.575 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:15:17.011 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:17.011+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:17.247 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:17.246+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:18.012 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:18.011+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:18.270 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:18.269+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:18.982 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:18.981+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:19.225 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:19.224+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:19.956 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:19.956+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:20.250 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:20.249+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:20.933 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:20.932+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:21.279 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:21.278+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:21.887 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:21.886+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:22.275 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:22.275+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:22.859 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:22.858+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:23.289 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:23.288+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:23.891 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:23.891+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:24.288 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:24.288+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:24.857 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:24.856+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:25.252 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:25.251+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:25.830 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:25.829+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:26.244 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:26.244+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:26.837 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:26.836+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:27.275 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:27.274+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:27.828 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:27.827+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:28.247 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:28.246+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:28.793 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:28.791+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:29.202 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:29.201+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:29.785 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:29.784+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:30.218 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:30.217+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:30.773 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:30.771+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:31.239 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:31.238+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:31.755 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:31.754+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:32.267 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:32.265+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:32.774 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:32.772+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:33.284 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:33.282+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:33.816 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:33.814+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:34.270 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:34.268+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:34.781 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:34.780+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:35.234 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:35.232+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:35.767 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:35.765+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:36.228 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:36.226+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:36.576 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:15:36.754 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:36.753+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:36.901 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:15:36.904 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:15:36.914 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":5831,"stamp":"2023-10-06T00:15:36.104200+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":296,"num_omap_keys":33,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24967,"ondisk_log_size":24967,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001986"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":1926,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050211+0000","last_change":"2023-10-05T21:11:08.661112+0000","last_active":"2023-10-05T22:09:12.050211+0000","last_peered":"2023-10-05T22:09:12.050211+0000","last_clean":"2023-10-05T22:09:12.050211+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-05T22:09:12.050211+0000","last_undegraded":"2023-10-05T22:09:12.050211+0000","last_fullsized":"2023-10-05T22:09:12.050211+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T00:45:46.698848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1829,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050119+0000","last_change":"2023-10-05T21:11:08.660937+0000","last_active":"2023-10-05T22:09:12.050119+0000","last_peered":"2023-10-05T22:09:12.050119+0000","last_clean":"2023-10-05T22:09:12.050119+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-05T22:09:12.050119+0000","last_undegraded":"2023-10-05T22:09:12.050119+0000","last_fullsized":"2023-10-05T22:09:12.050119+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:22.382648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"158'775","reported_seq":1644,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050024+0000","last_change":"2023-10-05T21:11:08.660871+0000","last_active":"2023-10-05T22:09:12.050024+0000","last_peered":"2023-10-05T22:09:12.050024+0000","last_clean":"2023-10-05T22:09:12.050024+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-05T22:09:12.050024+0000","last_undegraded":"2023-10-05T22:09:12.050024+0000","last_fullsized":"2023-10-05T22:09:12.050024+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:17.807335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"158'3217","reported_seq":8586,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050627+0000","last_change":"2023-10-05T21:11:08.651828+0000","last_active":"2023-10-05T22:09:12.050627+0000","last_peered":"2023-10-05T22:09:12.050627+0000","last_clean":"2023-10-05T22:09:12.050627+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-05T22:09:12.050627+0000","last_undegraded":"2023-10-05T22:09:12.050627+0000","last_fullsized":"2023-10-05T22:09:12.050627+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":3217,"log_dups_size":0,"ondisk_log_size":3217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:52:36.783122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11081,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049738+0000","last_change":"2023-10-05T21:11:08.661771+0000","last_active":"2023-10-05T22:09:12.049738+0000","last_peered":"2023-10-05T22:09:12.049738+0000","last_clean":"2023-10-05T22:09:12.049738+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-05T22:09:12.049738+0000","last_undegraded":"2023-10-05T22:09:12.049738+0000","last_fullsized":"2023-10-05T22:09:12.049738+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:30:11.617699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":281,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049561+0000","last_change":"2023-10-05T21:06:29.922199+0000","last_active":"2023-10-05T22:09:12.049561+0000","last_peered":"2023-10-05T22:09:12.049561+0000","last_clean":"2023-10-05T22:09:12.049561+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-05T22:09:12.049561+0000","last_undegraded":"2023-10-05T22:09:12.049561+0000","last_fullsized":"2023-10-05T22:09:12.049561+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:28.614917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:28.614917+0000","last_clean_scrub_stamp":"2023-10-05T21:06:28.614917+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:59:56.310688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2345,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049831+0000","last_change":"2023-10-05T21:11:08.659625+0000","last_active":"2023-10-05T22:09:12.049831+0000","last_peered":"2023-10-05T22:09:12.049831+0000","last_clean":"2023-10-05T22:09:12.049831+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-05T22:09:12.049831+0000","last_undegraded":"2023-10-05T22:09:12.049831+0000","last_fullsized":"2023-10-05T22:09:12.049831+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-06T22:30:43.667801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11729,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049612+0000","last_change":"2023-10-05T21:11:08.661262+0000","last_active":"2023-10-05T22:09:12.049612+0000","last_peered":"2023-10-05T22:09:12.049612+0000","last_clean":"2023-10-05T22:09:12.049612+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-05T22:09:12.049612+0000","last_undegraded":"2023-10-05T22:09:12.049612+0000","last_fullsized":"2023-10-05T22:09:12.049612+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-06T21:11:58.444833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11611,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050624+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-05T22:09:12.050624+0000","last_peered":"2023-10-05T22:09:12.050624+0000","last_clean":"2023-10-05T22:09:12.050624+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-05T22:09:12.050624+0000","last_undegraded":"2023-10-05T22:09:12.050624+0000","last_fullsized":"2023-10-05T22:09:12.050624+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-06T22:30:11.666110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1781,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050624+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-05T22:09:12.050624+0000","last_peered":"2023-10-05T22:09:12.050624+0000","last_clean":"2023-10-05T22:09:12.050624+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-05T22:09:12.050624+0000","last_undegraded":"2023-10-05T22:09:12.050624+0000","last_fullsized":"2023-10-05T22:09:12.050624+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T01:49:48.848559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1750,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049927+0000","last_change":"2023-10-05T21:11:08.661368+0000","last_active":"2023-10-05T22:09:12.049927+0000","last_peered":"2023-10-05T22:09:12.049927+0000","last_clean":"2023-10-05T22:09:12.049927+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-05T22:09:12.049927+0000","last_undegraded":"2023-10-05T22:09:12.049927+0000","last_fullsized":"2023-10-05T22:09:12.049927+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T01:16:04.809466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17193,"ondisk_log_size":17193,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":296,"num_omap_keys":33,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7677,"ondisk_log_size":7677,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740647,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:14:59 2023","interfaces":[{"interface":"back","average":{"1min":0.21199999999999999,"5min":0.217,"15min":0.217},"min":{"1min":0.16400000000000001,"5min":0.155,"15min":0.14399999999999999},"max":{"1min":0.32100000000000001,"5min":0.45900000000000002,"15min":0.46999999999999997},"last":1.153},{"interface":"front","average":{"1min":0.23100000000000001,"5min":0.23899999999999999,"15min":0.24299999999999999},"min":{"1min":0.17799999999999999,"5min":0.17599999999999999,"15min":0.16700000000000001},"max":{"1min":0.41899999999999998,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.308}]},{"osd":1,"last update":"Fri Oct 6 00:14:59 2023","interfaces":[{"interface":"back","average":{"1min":0.248,"5min":0.249,"15min":0.24199999999999999},"min":{"1min":0.191,"5min":0.17299999999999999,"15min":0.153},"max":{"1min":0.32100000000000001,"5min":0.45400000000000001,"15min":0.48699999999999999},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.26400000000000001,"5min":0.27000000000000002,"15min":0.26900000000000002},"min":{"1min":0.223,"5min":0.19600000000000001,"15min":0.16300000000000001},"max":{"1min":0.32400000000000001,"5min":0.442,"15min":0.51200000000000001},"last":0.35299999999999998}]}]},{"osd":0,"up_from":8,"seq":34359740648,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.19500000000000001,"5min":0.191,"15min":0.20000000000000001},"min":{"1min":0.13600000000000001,"5min":0.128,"15min":0.128},"max":{"1min":0.309,"5min":0.34999999999999998,"15min":0.439},"last":0.16400000000000001},{"interface":"front","average":{"1min":0.26800000000000002,"5min":0.26600000000000001,"15min":0.26200000000000001},"min":{"1min":0.189,"5min":0.189,"15min":0.152},"max":{"1min":0.33200000000000002,"5min":0.434,"15min":0.47899999999999998},"last":0.25600000000000001}]},{"osd":2,"last update":"Fri Oct 6 00:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.22800000000000001,"5min":0.23400000000000001,"15min":0.23100000000000001},"min":{"1min":0.16700000000000001,"5min":0.16300000000000001,"15min":0.16300000000000001},"max":{"1min":0.30399999999999999,"5min":0.36799999999999999,"15min":0.439},"last":0.216},{"interface":"front","average":{"1min":0.28699999999999998,"5min":0.29599999999999999,"15min":0.29399999999999998},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.35599999999999998,"5min":0.46400000000000002,"15min":0.53600000000000003},"last":0.29099999999999998}]}]},{"osd":1,"up_from":8,"seq":34359740643,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:15:27 2023","interfaces":[{"interface":"back","average":{"1min":0.223,"5min":0.20999999999999999,"15min":0.20699999999999999},"min":{"1min":0.159,"5min":0.13100000000000001,"15min":0.13},"max":{"1min":0.33600000000000002,"5min":0.497,"15min":0.497},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.26200000000000001,"5min":0.26600000000000001,"15min":0.27300000000000002},"min":{"1min":0.16800000000000001,"5min":0.154,"15min":0.154},"max":{"1min":0.41099999999999998,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.35299999999999998}]},{"osd":2,"last update":"Fri Oct 6 00:15:27 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.23200000000000001,"15min":0.23499999999999999},"min":{"1min":0.186,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":0.37,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.317},{"interface":"front","average":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27500000000000002},"min":{"1min":0.223,"5min":0.20100000000000001,"15min":0.19900000000000001},"max":{"1min":0.39300000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.39000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:15:36.916 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:15:36.916 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:15:36.916 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:15:36.917 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:15:36.917 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:15:36.917 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-05T21:06:28.614917+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=28, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:15:36.917 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:15:36.917 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:15:36.918 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:15:36.918 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:15:36.918 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:15:36.918 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:15:37.196 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:37.195+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:37.731 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:37.730+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:38.242 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:38.241+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:38.703 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:38.701+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:39.227 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:39.226+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:39.681 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:39.680+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:40.249 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:40.248+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:40.699 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:40.698+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:41.257 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:41.256+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:41.692 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:41.691+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:42.235 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:42.233+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:42.711 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:42.709+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:43.234 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:43.232+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:43.692 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:43.690+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:44.227 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:44.225+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:44.663 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:44.662+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:45.185 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:45.183+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:45.689 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:45.688+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:46.235 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:46.234+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:46.710 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:46.709+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:47.266 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:47.265+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:47.740 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:47.738+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:48.254 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:48.252+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:48.781 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:48.780+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:49.215 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:49.214+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:49.820 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:49.818+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:50.248 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:50.245+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:50.812 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:50.810+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:51.276 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:51.274+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:51.806 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:51.804+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:52.283 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:52.282+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:52.854 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:52.852+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:53.339 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:53.268+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:53.875 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:53.874+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:54.299 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:54.298+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:54.925 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:54.923+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:55.269 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:55.268+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:55.964 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:55.962+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:56.303 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:56.301+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:56.919 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:15:56.965 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:56.963+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:57.287 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:15:57.353 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:15:57.353 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":5841,"stamp":"2023-10-06T00:15:56.107403+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":296,"num_omap_keys":33,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24967,"ondisk_log_size":24967,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001887"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":1926,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050211+0000","last_change":"2023-10-05T21:11:08.661112+0000","last_active":"2023-10-05T22:09:12.050211+0000","last_peered":"2023-10-05T22:09:12.050211+0000","last_clean":"2023-10-05T22:09:12.050211+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-05T22:09:12.050211+0000","last_undegraded":"2023-10-05T22:09:12.050211+0000","last_fullsized":"2023-10-05T22:09:12.050211+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T00:45:46.698848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1829,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050119+0000","last_change":"2023-10-05T21:11:08.660937+0000","last_active":"2023-10-05T22:09:12.050119+0000","last_peered":"2023-10-05T22:09:12.050119+0000","last_clean":"2023-10-05T22:09:12.050119+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-05T22:09:12.050119+0000","last_undegraded":"2023-10-05T22:09:12.050119+0000","last_fullsized":"2023-10-05T22:09:12.050119+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:22.382648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"158'775","reported_seq":1644,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050024+0000","last_change":"2023-10-05T21:11:08.660871+0000","last_active":"2023-10-05T22:09:12.050024+0000","last_peered":"2023-10-05T22:09:12.050024+0000","last_clean":"2023-10-05T22:09:12.050024+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-05T22:09:12.050024+0000","last_undegraded":"2023-10-05T22:09:12.050024+0000","last_fullsized":"2023-10-05T22:09:12.050024+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:17.807335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"158'3217","reported_seq":8586,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050627+0000","last_change":"2023-10-05T21:11:08.651828+0000","last_active":"2023-10-05T22:09:12.050627+0000","last_peered":"2023-10-05T22:09:12.050627+0000","last_clean":"2023-10-05T22:09:12.050627+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-05T22:09:12.050627+0000","last_undegraded":"2023-10-05T22:09:12.050627+0000","last_fullsized":"2023-10-05T22:09:12.050627+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":3217,"log_dups_size":0,"ondisk_log_size":3217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:52:36.783122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11081,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049738+0000","last_change":"2023-10-05T21:11:08.661771+0000","last_active":"2023-10-05T22:09:12.049738+0000","last_peered":"2023-10-05T22:09:12.049738+0000","last_clean":"2023-10-05T22:09:12.049738+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-05T22:09:12.049738+0000","last_undegraded":"2023-10-05T22:09:12.049738+0000","last_fullsized":"2023-10-05T22:09:12.049738+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:30:11.617699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":281,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049561+0000","last_change":"2023-10-05T21:06:29.922199+0000","last_active":"2023-10-05T22:09:12.049561+0000","last_peered":"2023-10-05T22:09:12.049561+0000","last_clean":"2023-10-05T22:09:12.049561+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-05T22:09:12.049561+0000","last_undegraded":"2023-10-05T22:09:12.049561+0000","last_fullsized":"2023-10-05T22:09:12.049561+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:28.614917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:28.614917+0000","last_clean_scrub_stamp":"2023-10-05T21:06:28.614917+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:59:56.310688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2345,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049831+0000","last_change":"2023-10-05T21:11:08.659625+0000","last_active":"2023-10-05T22:09:12.049831+0000","last_peered":"2023-10-05T22:09:12.049831+0000","last_clean":"2023-10-05T22:09:12.049831+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-05T22:09:12.049831+0000","last_undegraded":"2023-10-05T22:09:12.049831+0000","last_fullsized":"2023-10-05T22:09:12.049831+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-06T22:30:43.667801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11729,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049612+0000","last_change":"2023-10-05T21:11:08.661262+0000","last_active":"2023-10-05T22:09:12.049612+0000","last_peered":"2023-10-05T22:09:12.049612+0000","last_clean":"2023-10-05T22:09:12.049612+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-05T22:09:12.049612+0000","last_undegraded":"2023-10-05T22:09:12.049612+0000","last_fullsized":"2023-10-05T22:09:12.049612+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-06T21:11:58.444833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11611,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050624+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-05T22:09:12.050624+0000","last_peered":"2023-10-05T22:09:12.050624+0000","last_clean":"2023-10-05T22:09:12.050624+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-05T22:09:12.050624+0000","last_undegraded":"2023-10-05T22:09:12.050624+0000","last_fullsized":"2023-10-05T22:09:12.050624+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-06T22:30:11.666110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1781,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.050624+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-05T22:09:12.050624+0000","last_peered":"2023-10-05T22:09:12.050624+0000","last_clean":"2023-10-05T22:09:12.050624+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-05T22:09:12.050624+0000","last_undegraded":"2023-10-05T22:09:12.050624+0000","last_fullsized":"2023-10-05T22:09:12.050624+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T01:49:48.848559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1750,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-05T22:09:12.049927+0000","last_change":"2023-10-05T21:11:08.661368+0000","last_active":"2023-10-05T22:09:12.049927+0000","last_peered":"2023-10-05T22:09:12.049927+0000","last_clean":"2023-10-05T22:09:12.049927+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-05T22:09:12.049927+0000","last_undegraded":"2023-10-05T22:09:12.049927+0000","last_fullsized":"2023-10-05T22:09:12.049927+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T01:16:04.809466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17193,"ondisk_log_size":17193,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":296,"num_omap_keys":33,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7677,"ondisk_log_size":7677,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740651,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:14:59 2023","interfaces":[{"interface":"back","average":{"1min":0.21199999999999999,"5min":0.217,"15min":0.217},"min":{"1min":0.16400000000000001,"5min":0.155,"15min":0.14399999999999999},"max":{"1min":0.32100000000000001,"5min":0.45900000000000002,"15min":0.46999999999999997},"last":0.16200000000000001},{"interface":"front","average":{"1min":0.23100000000000001,"5min":0.23899999999999999,"15min":0.24299999999999999},"min":{"1min":0.17799999999999999,"5min":0.17599999999999999,"15min":0.16700000000000001},"max":{"1min":0.41899999999999998,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.185}]},{"osd":1,"last update":"Fri Oct 6 00:14:59 2023","interfaces":[{"interface":"back","average":{"1min":0.248,"5min":0.249,"15min":0.24199999999999999},"min":{"1min":0.191,"5min":0.17299999999999999,"15min":0.153},"max":{"1min":0.32100000000000001,"5min":0.45400000000000001,"15min":0.48699999999999999},"last":0.17199999999999999},{"interface":"front","average":{"1min":0.26400000000000001,"5min":0.27000000000000002,"15min":0.26900000000000002},"min":{"1min":0.223,"5min":0.19600000000000001,"15min":0.16300000000000001},"max":{"1min":0.32400000000000001,"5min":0.442,"15min":0.51200000000000001},"last":0.23100000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740652,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.19500000000000001,"5min":0.191,"15min":0.20000000000000001},"min":{"1min":0.13600000000000001,"5min":0.128,"15min":0.128},"max":{"1min":0.309,"5min":0.34999999999999998,"15min":0.439},"last":0.17299999999999999},{"interface":"front","average":{"1min":0.26800000000000002,"5min":0.26600000000000001,"15min":0.26200000000000001},"min":{"1min":0.189,"5min":0.189,"15min":0.152},"max":{"1min":0.33200000000000002,"5min":0.434,"15min":0.47899999999999998},"last":0.29199999999999998}]},{"osd":2,"last update":"Fri Oct 6 00:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.22800000000000001,"5min":0.23400000000000001,"15min":0.23100000000000001},"min":{"1min":0.16700000000000001,"5min":0.16300000000000001,"15min":0.16300000000000001},"max":{"1min":0.30399999999999999,"5min":0.36799999999999999,"15min":0.439},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.28699999999999998,"5min":0.29599999999999999,"15min":0.29399999999999998},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.35599999999999998,"5min":0.46400000000000002,"15min":0.53600000000000003},"last":0.318}]}]},{"osd":1,"up_from":8,"seq":34359740647,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:15:27 2023","interfaces":[{"interface":"back","average":{"1min":0.223,"5min":0.20999999999999999,"15min":0.20699999999999999},"min":{"1min":0.159,"5min":0.13100000000000001,"15min":0.13},"max":{"1min":0.33600000000000002,"5min":0.497,"15min":0.497},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.26200000000000001,"5min":0.26600000000000001,"15min":0.27300000000000002},"min":{"1min":0.16800000000000001,"5min":0.154,"15min":0.154},"max":{"1min":0.41099999999999998,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.47599999999999998}]},{"osd":2,"last update":"Fri Oct 6 00:15:27 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.23200000000000001,"15min":0.23499999999999999},"min":{"1min":0.186,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":0.37,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.437},{"interface":"front","average":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27500000000000002},"min":{"1min":0.223,"5min":0.20100000000000001,"15min":0.19900000000000001},"max":{"1min":0.39300000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.50900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:15:57.356 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:15:57.356 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:15:57.356 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:15:57.356 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:15:57.357 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:15:57.357 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-05T21:06:28.614917+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=28, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:15:57.357 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:15:57.357 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:15:57.357 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:15:57.358 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:15:57.358 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:15:57.358 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.7 2023-10-06T00:15:57.358 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:57.320+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:57.738 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 2.7 on osd.1 to deep-scrub 2023-10-06T00:15:57.812 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.6 2023-10-06T00:15:57.970 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:57.968+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:58.348 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 2.6 on osd.1 to deep-scrub 2023-10-06T00:15:58.349 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.5 2023-10-06T00:15:58.349 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:58.292+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:58.685 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 2.5 on osd.1 to deep-scrub 2023-10-06T00:15:58.696 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.1 2023-10-06T00:15:58.967 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:58.965+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:59.038 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 3.1 on osd.0 to deep-scrub 2023-10-06T00:15:59.052 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.2 2023-10-06T00:15:59.255 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:15:59.253+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:15:59.396 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 3.2 on osd.1 to deep-scrub 2023-10-06T00:15:59.408 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 1.0 2023-10-06T00:15:59.780 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 1.0 on osd.1 to deep-scrub 2023-10-06T00:15:59.792 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.3 2023-10-06T00:15:59.992 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:15:59.990+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:00.166 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 2.3 on osd.1 to deep-scrub 2023-10-06T00:16:00.177 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.0 2023-10-06T00:16:00.296 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:00.294+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:00.518 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 3.0 on osd.1 to deep-scrub 2023-10-06T00:16:00.529 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.3 2023-10-06T00:16:00.899 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 3.3 on osd.0 to deep-scrub 2023-10-06T00:16:00.911 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.2 2023-10-06T00:16:00.950 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:00.949+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:01.250 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 2.2 on osd.0 to deep-scrub 2023-10-06T00:16:01.261 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.4 2023-10-06T00:16:01.336 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:01.335+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:01.622 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 2.4 on osd.1 to deep-scrub 2023-10-06T00:16:01.635 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:16:01.937 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:01.936+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:02.336 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:02.335+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:02.971 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:02.969+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:03.376 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:03.374+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:03.959 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:03.957+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:04.381 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:04.379+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:04.941 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:04.939+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:05.335 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:05.334+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:05.947 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:05.945+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:06.364 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:06.363+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:06.983 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:06.982+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:07.380 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:07.379+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:08.024 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:08.023+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:08.366 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:08.365+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:09.061 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:09.061+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:09.381 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:09.380+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:10.086 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:10.085+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:10.429 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:10.429+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:11.113 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:11.113+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:11.385 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:11.384+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:12.153 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:12.152+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:12.340 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:12.339+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:13.154 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:13.153+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:13.374 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:13.373+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:14.129 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:14.128+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:14.420 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:14.419+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:15.089 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:15.088+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:15.419 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:15.418+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:16.114 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:16.113+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:16.379 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:16.378+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:17.160 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:17.159+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:17.369 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:17.368+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:18.195 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:18.195+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:18.333 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:18.332+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:19.201 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:19.200+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:19.366 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:19.365+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:20.248 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:20.247+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:20.403 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:20.403+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:21.252 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:21.251+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:21.421 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:21.421+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:21.635 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:16:21.974 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:16:21.977 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:16:21.989 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":5853,"stamp":"2023-10-06T00:16:20.111042+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001862"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":1930,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:16:08.023657+0000","last_change":"2023-10-06T00:16:08.023657+0000","last_active":"2023-10-06T00:16:08.023657+0000","last_peered":"2023-10-06T00:16:08.023657+0000","last_clean":"2023-10-06T00:16:08.023657+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:16:08.023657+0000","last_undegraded":"2023-10-06T00:16:08.023657+0000","last_fullsized":"2023-10-06T00:16:08.023657+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1833,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:16:06.982559+0000","last_change":"2023-10-06T00:16:06.982559+0000","last_active":"2023-10-06T00:16:06.982559+0000","last_peered":"2023-10-06T00:16:06.982559+0000","last_clean":"2023-10-06T00:16:06.982559+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:16:06.982559+0000","last_undegraded":"2023-10-06T00:16:06.982559+0000","last_fullsized":"2023-10-06T00:16:06.982559+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8616,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:16:00.339983+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:16:00.339983+0000","last_peered":"2023-10-06T00:16:00.339983+0000","last_clean":"2023-10-06T00:16:00.339983+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:16:00.339983+0000","last_undegraded":"2023-10-06T00:16:00.339983+0000","last_fullsized":"2023-10-06T00:16:00.339983+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11085,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:16:12.152570+0000","last_change":"2023-10-06T00:16:12.152570+0000","last_active":"2023-10-06T00:16:12.152570+0000","last_peered":"2023-10-06T00:16:12.152570+0000","last_clean":"2023-10-06T00:16:12.152570+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:16:12.152570+0000","last_undegraded":"2023-10-06T00:16:12.152570+0000","last_fullsized":"2023-10-06T00:16:12.152570+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2349,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:16:04.940535+0000","last_change":"2023-10-06T00:16:04.940535+0000","last_active":"2023-10-06T00:16:04.940535+0000","last_peered":"2023-10-06T00:16:04.940535+0000","last_clean":"2023-10-06T00:16:04.940535+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:16:04.940535+0000","last_undegraded":"2023-10-06T00:16:04.940535+0000","last_fullsized":"2023-10-06T00:16:04.940535+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11733,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:16:10.085606+0000","last_change":"2023-10-06T00:16:10.085606+0000","last_active":"2023-10-06T00:16:10.085606+0000","last_peered":"2023-10-06T00:16:10.085606+0000","last_clean":"2023-10-06T00:16:10.085606+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:16:10.085606+0000","last_undegraded":"2023-10-06T00:16:10.085606+0000","last_fullsized":"2023-10-06T00:16:10.085606+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11612,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-05T22:09:47.054310+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-05T22:09:47.054310+0000","last_peered":"2023-10-05T22:09:47.054310+0000","last_clean":"2023-10-05T22:09:47.054310+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-05T22:09:47.054310+0000","last_undegraded":"2023-10-05T22:09:47.054310+0000","last_fullsized":"2023-10-05T22:09:47.054310+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1782,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-05T22:09:47.054518+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-05T22:09:47.054518+0000","last_peered":"2023-10-05T22:09:47.054518+0000","last_clean":"2023-10-05T22:09:47.054518+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-05T22:09:47.054518+0000","last_undegraded":"2023-10-05T22:09:47.054518+0000","last_fullsized":"2023-10-05T22:09:47.054518+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1757,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:16:15.088630+0000","last_change":"2023-10-06T00:16:15.088630+0000","last_active":"2023-10-06T00:16:15.088630+0000","last_peered":"2023-10-06T00:16:15.088630+0000","last_clean":"2023-10-06T00:16:15.088630+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:16:15.088630+0000","last_undegraded":"2023-10-06T00:16:15.088630+0000","last_fullsized":"2023-10-06T00:16:15.088630+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740656,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:16:02 2023","interfaces":[{"interface":"back","average":{"1min":0.27800000000000002,"5min":0.23100000000000001,"15min":0.223},"min":{"1min":0.16200000000000001,"5min":0.155,"15min":0.14599999999999999},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.193},{"interface":"front","average":{"1min":0.249,"5min":0.24099999999999999,"15min":0.245},"min":{"1min":0.185,"5min":0.17599999999999999,"15min":0.16700000000000001},"max":{"1min":0.40899999999999997,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.22600000000000001}]},{"osd":1,"last update":"Fri Oct 6 00:16:02 2023","interfaces":[{"interface":"back","average":{"1min":0.26100000000000001,"5min":0.253,"15min":0.24399999999999999},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.153},"max":{"1min":0.39800000000000002,"5min":0.45400000000000001,"15min":0.48699999999999999},"last":0.246},{"interface":"front","average":{"1min":0.29499999999999998,"5min":0.27600000000000002,"15min":0.27200000000000002},"min":{"1min":0.23100000000000001,"5min":0.19600000000000001,"15min":0.16300000000000001},"max":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.51200000000000001},"last":0.29399999999999998}]}]},{"osd":0,"up_from":8,"seq":34359740657,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:16:06 2023","interfaces":[{"interface":"back","average":{"1min":0.215,"5min":0.19800000000000001,"15min":0.20000000000000001},"min":{"1min":0.157,"5min":0.128,"15min":0.128},"max":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.41299999999999998},"last":0.14399999999999999},{"interface":"front","average":{"1min":0.28399999999999997,"5min":0.27300000000000002,"15min":0.26400000000000001},"min":{"1min":0.214,"5min":0.189,"15min":0.152},"max":{"1min":0.443,"5min":0.443,"15min":0.47899999999999998},"last":0.186}]},{"osd":2,"last update":"Fri Oct 6 00:16:06 2023","interfaces":[{"interface":"back","average":{"1min":0.25,"5min":0.23999999999999999,"15min":0.23200000000000001},"min":{"1min":0.183,"5min":0.16300000000000001,"15min":0.16300000000000001},"max":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.439},"last":0.217},{"interface":"front","average":{"1min":0.313,"5min":0.30199999999999999,"15min":0.29599999999999999},"min":{"1min":0.249,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.53600000000000003},"last":0.245}]}]},{"osd":1,"up_from":8,"seq":34359740652,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:15:27 2023","interfaces":[{"interface":"back","average":{"1min":0.223,"5min":0.20999999999999999,"15min":0.20699999999999999},"min":{"1min":0.159,"5min":0.13100000000000001,"15min":0.13},"max":{"1min":0.33600000000000002,"5min":0.497,"15min":0.497},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.26200000000000001,"5min":0.26600000000000001,"15min":0.27300000000000002},"min":{"1min":0.16800000000000001,"5min":0.154,"15min":0.154},"max":{"1min":0.41099999999999998,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.16300000000000001}]},{"osd":2,"last update":"Fri Oct 6 00:15:27 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.23200000000000001,"15min":0.23499999999999999},"min":{"1min":0.186,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":0.37,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.19700000000000001},{"interface":"front","average":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27500000000000002},"min":{"1min":0.223,"5min":0.20100000000000001,"15min":0.19900000000000001},"max":{"1min":0.39300000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.219}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:16:21.990 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:16:21.991 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:16:21.991 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:16:21.991 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:16:21.991 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:16:21.991 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:16:21.992 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:16:21.992 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:16:21.992 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:16:21.992 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:16:22.287 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:22.287+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:22.461 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:22.461+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:23.292 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:23.291+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:23.458 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:23.457+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:24.276 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:24.275+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:24.498 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:24.498+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:25.287 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:25.286+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:25.504 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:25.504+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:26.336 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:26.335+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:26.507 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:26.505+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:27.365 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:27.363+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:27.526 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:27.524+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:28.401 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:28.400+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:28.541 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:28.540+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:29.399 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:29.397+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:29.513 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:29.511+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:30.362 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:30.360+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:30.477 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:30.475+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:31.356 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:31.355+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:31.460 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:31.458+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:32.309 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:32.308+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:32.471 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:32.469+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:33.316 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:33.314+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:33.481 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:33.479+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:34.297 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:34.296+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:34.529 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:34.527+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:35.267 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:35.266+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:35.530 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:35.528+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:36.256 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:36.255+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:36.544 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:36.542+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:37.294 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:37.293+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:37.558 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:37.556+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:38.305 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:38.304+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:38.564 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:38.563+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:39.344 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:39.342+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:39.596 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:39.595+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:40.377 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:40.376+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:40.634 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:40.632+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:41.377 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:41.375+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:41.652 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:41.650+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:41.993 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:16:42.319 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:16:42.321 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:16:42.333 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":5864,"stamp":"2023-10-06T00:16:42.114627+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001897"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":1933,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:16:24.275452+0000","last_change":"2023-10-06T00:16:24.275452+0000","last_active":"2023-10-06T00:16:24.275452+0000","last_peered":"2023-10-06T00:16:24.275452+0000","last_clean":"2023-10-06T00:16:24.275452+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:16:24.275452+0000","last_undegraded":"2023-10-06T00:16:24.275452+0000","last_fullsized":"2023-10-06T00:16:24.275452+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1836,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:16:21.251732+0000","last_change":"2023-10-06T00:16:21.251732+0000","last_active":"2023-10-06T00:16:21.251732+0000","last_peered":"2023-10-06T00:16:21.251732+0000","last_clean":"2023-10-06T00:16:21.251732+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:16:21.251732+0000","last_undegraded":"2023-10-06T00:16:21.251732+0000","last_fullsized":"2023-10-06T00:16:21.251732+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8616,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:16:00.339983+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:16:00.339983+0000","last_peered":"2023-10-06T00:16:00.339983+0000","last_clean":"2023-10-06T00:16:00.339983+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:16:00.339983+0000","last_undegraded":"2023-10-06T00:16:00.339983+0000","last_fullsized":"2023-10-06T00:16:00.339983+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11088,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:16:29.398641+0000","last_change":"2023-10-06T00:16:29.398641+0000","last_active":"2023-10-06T00:16:29.398641+0000","last_peered":"2023-10-06T00:16:29.398641+0000","last_clean":"2023-10-06T00:16:29.398641+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:16:29.398641+0000","last_undegraded":"2023-10-06T00:16:29.398641+0000","last_fullsized":"2023-10-06T00:16:29.398641+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2355,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:16:31.356210+0000","last_change":"2023-10-06T00:16:31.356210+0000","last_active":"2023-10-06T00:16:31.356210+0000","last_peered":"2023-10-06T00:16:31.356210+0000","last_clean":"2023-10-06T00:16:31.356210+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:16:31.356210+0000","last_undegraded":"2023-10-06T00:16:31.356210+0000","last_fullsized":"2023-10-06T00:16:31.356210+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11736,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:16:27.364659+0000","last_change":"2023-10-06T00:16:27.364659+0000","last_active":"2023-10-06T00:16:27.364659+0000","last_peered":"2023-10-06T00:16:27.364659+0000","last_clean":"2023-10-06T00:16:27.364659+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:16:27.364659+0000","last_undegraded":"2023-10-06T00:16:27.364659+0000","last_fullsized":"2023-10-06T00:16:27.364659+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11612,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-05T22:09:47.054310+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-05T22:09:47.054310+0000","last_peered":"2023-10-05T22:09:47.054310+0000","last_clean":"2023-10-05T22:09:47.054310+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-05T22:09:47.054310+0000","last_undegraded":"2023-10-05T22:09:47.054310+0000","last_fullsized":"2023-10-05T22:09:47.054310+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1782,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-05T22:09:47.054518+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-05T22:09:47.054518+0000","last_peered":"2023-10-05T22:09:47.054518+0000","last_clean":"2023-10-05T22:09:47.054518+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-05T22:09:47.054518+0000","last_undegraded":"2023-10-05T22:09:47.054518+0000","last_fullsized":"2023-10-05T22:09:47.054518+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1760,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:16:30.361690+0000","last_change":"2023-10-06T00:16:30.361690+0000","last_active":"2023-10-06T00:16:30.361690+0000","last_peered":"2023-10-06T00:16:30.361690+0000","last_clean":"2023-10-06T00:16:30.361690+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:16:30.361690+0000","last_undegraded":"2023-10-06T00:16:30.361690+0000","last_fullsized":"2023-10-06T00:16:30.361690+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740660,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:16:02 2023","interfaces":[{"interface":"back","average":{"1min":0.27800000000000002,"5min":0.23100000000000001,"15min":0.223},"min":{"1min":0.16200000000000001,"5min":0.155,"15min":0.14599999999999999},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.19},{"interface":"front","average":{"1min":0.249,"5min":0.24099999999999999,"15min":0.245},"min":{"1min":0.185,"5min":0.17599999999999999,"15min":0.16700000000000001},"max":{"1min":0.40899999999999997,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.217}]},{"osd":1,"last update":"Fri Oct 6 00:16:02 2023","interfaces":[{"interface":"back","average":{"1min":0.26100000000000001,"5min":0.253,"15min":0.24399999999999999},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.153},"max":{"1min":0.39800000000000002,"5min":0.45400000000000001,"15min":0.48699999999999999},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.29499999999999998,"5min":0.27600000000000002,"15min":0.27200000000000002},"min":{"1min":0.23100000000000001,"5min":0.19600000000000001,"15min":0.16300000000000001},"max":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.51200000000000001},"last":0.247}]}]},{"osd":0,"up_from":8,"seq":34359740661,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:16:06 2023","interfaces":[{"interface":"back","average":{"1min":0.215,"5min":0.19800000000000001,"15min":0.20000000000000001},"min":{"1min":0.157,"5min":0.128,"15min":0.128},"max":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.41299999999999998},"last":0.20200000000000001},{"interface":"front","average":{"1min":0.28399999999999997,"5min":0.27300000000000002,"15min":0.26400000000000001},"min":{"1min":0.214,"5min":0.189,"15min":0.152},"max":{"1min":0.443,"5min":0.443,"15min":0.47899999999999998},"last":0.26200000000000001}]},{"osd":2,"last update":"Fri Oct 6 00:16:06 2023","interfaces":[{"interface":"back","average":{"1min":0.25,"5min":0.23999999999999999,"15min":0.23200000000000001},"min":{"1min":0.183,"5min":0.16300000000000001,"15min":0.16300000000000001},"max":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.439},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.313,"5min":0.30199999999999999,"15min":0.29599999999999999},"min":{"1min":0.249,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.53600000000000003},"last":0.28599999999999998}]}]},{"osd":1,"up_from":8,"seq":34359740656,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.23899999999999999,"5min":0.218,"15min":0.20899999999999999},"min":{"1min":0.152,"5min":0.13100000000000001,"15min":0.13},"max":{"1min":0.41399999999999998,"5min":0.497,"15min":0.497},"last":0.17199999999999999},{"interface":"front","average":{"1min":0.28000000000000003,"5min":0.26800000000000002,"15min":0.27300000000000002},"min":{"1min":0.16300000000000001,"5min":0.154,"15min":0.154},"max":{"1min":0.47599999999999998,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.26600000000000001}]},{"osd":2,"last update":"Fri Oct 6 00:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.252,"5min":0.23699999999999999,"15min":0.23499999999999999},"min":{"1min":0.17599999999999999,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":0.437,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.28100000000000003,"5min":0.27400000000000002,"15min":0.27500000000000002},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.19900000000000001},"max":{"1min":0.50900000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.20599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:16:42.336 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:16:42.336 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:16:42.337 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:16:42.337 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:16:42.337 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:16:42.338 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:16:42.338 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:16:42.338 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:16:42.338 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:16:42.339 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:16:42.382 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:42.380+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:42.656 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:42.654+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:43.392 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:43.390+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:43.659 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:43.658+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:44.371 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:44.370+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:44.658 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:44.656+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:45.390 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:45.388+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:45.639 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:45.638+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:46.393 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:46.391+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:46.611 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:46.609+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:47.362 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:47.361+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:47.619 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:47.617+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:48.384 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:48.382+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:48.665 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:48.664+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:49.387 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:49.385+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:49.657 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:49.655+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:50.396 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:50.394+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:50.613 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:50.611+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:51.377 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:51.375+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:51.639 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:51.637+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:52.331 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:52.329+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:52.634 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:52.633+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:53.288 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:53.287+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:53.625 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:53.623+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:54.281 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:54.279+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:54.621 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:54.619+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:55.302 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:55.300+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:55.658 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:55.656+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:56.329 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:56.327+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:56.615 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:56.613+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:57.300 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:57.298+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:57.649 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:57.648+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:58.251 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:58.250+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:58.653 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:58.652+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:59.233 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:16:59.231+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:16:59.681 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:16:59.680+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:00.245 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:00.244+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:00.688 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:00.686+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:01.286 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:01.285+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:01.703 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:01.701+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:02.249 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:02.248+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:02.339 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:17:02.669 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:17:02.671 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:17:02.683 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":5874,"stamp":"2023-10-06T00:17:02.117632+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001939"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":1939,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:16:55.301426+0000","last_change":"2023-10-06T00:16:55.301426+0000","last_active":"2023-10-06T00:16:55.301426+0000","last_peered":"2023-10-06T00:16:55.301426+0000","last_clean":"2023-10-06T00:16:55.301426+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:16:55.301426+0000","last_undegraded":"2023-10-06T00:16:55.301426+0000","last_fullsized":"2023-10-06T00:16:55.301426+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1842,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:16:51.376534+0000","last_change":"2023-10-06T00:16:51.376534+0000","last_active":"2023-10-06T00:16:51.376534+0000","last_peered":"2023-10-06T00:16:51.376534+0000","last_clean":"2023-10-06T00:16:51.376534+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:16:51.376534+0000","last_undegraded":"2023-10-06T00:16:51.376534+0000","last_fullsized":"2023-10-06T00:16:51.376534+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8616,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:16:00.339983+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:16:00.339983+0000","last_peered":"2023-10-06T00:16:00.339983+0000","last_clean":"2023-10-06T00:16:00.339983+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:16:00.339983+0000","last_undegraded":"2023-10-06T00:16:00.339983+0000","last_fullsized":"2023-10-06T00:16:00.339983+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11091,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:16:46.393043+0000","last_change":"2023-10-06T00:16:46.393043+0000","last_active":"2023-10-06T00:16:46.393043+0000","last_peered":"2023-10-06T00:16:46.393043+0000","last_clean":"2023-10-06T00:16:46.393043+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:16:46.393043+0000","last_undegraded":"2023-10-06T00:16:46.393043+0000","last_fullsized":"2023-10-06T00:16:46.393043+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2358,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:16:49.386514+0000","last_change":"2023-10-06T00:16:49.386514+0000","last_active":"2023-10-06T00:16:49.386514+0000","last_peered":"2023-10-06T00:16:49.386514+0000","last_clean":"2023-10-06T00:16:49.386514+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:16:49.386514+0000","last_undegraded":"2023-10-06T00:16:49.386514+0000","last_fullsized":"2023-10-06T00:16:49.386514+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11739,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:16:45.389743+0000","last_change":"2023-10-06T00:16:45.389743+0000","last_active":"2023-10-06T00:16:45.389743+0000","last_peered":"2023-10-06T00:16:45.389743+0000","last_clean":"2023-10-06T00:16:45.389743+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:16:45.389743+0000","last_undegraded":"2023-10-06T00:16:45.389743+0000","last_fullsized":"2023-10-06T00:16:45.389743+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11612,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-05T22:09:47.054310+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-05T22:09:47.054310+0000","last_peered":"2023-10-05T22:09:47.054310+0000","last_clean":"2023-10-05T22:09:47.054310+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-05T22:09:47.054310+0000","last_undegraded":"2023-10-05T22:09:47.054310+0000","last_fullsized":"2023-10-05T22:09:47.054310+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1782,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-05T22:09:47.054518+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-05T22:09:47.054518+0000","last_peered":"2023-10-05T22:09:47.054518+0000","last_clean":"2023-10-05T22:09:47.054518+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-05T22:09:47.054518+0000","last_undegraded":"2023-10-05T22:09:47.054518+0000","last_fullsized":"2023-10-05T22:09:47.054518+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1763,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:16:47.362263+0000","last_change":"2023-10-06T00:16:47.362263+0000","last_active":"2023-10-06T00:16:47.362263+0000","last_peered":"2023-10-06T00:16:47.362263+0000","last_clean":"2023-10-06T00:16:47.362263+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:16:47.362263+0000","last_undegraded":"2023-10-06T00:16:47.362263+0000","last_fullsized":"2023-10-06T00:16:47.362263+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740664,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:16:02 2023","interfaces":[{"interface":"back","average":{"1min":0.27800000000000002,"5min":0.23100000000000001,"15min":0.223},"min":{"1min":0.16200000000000001,"5min":0.155,"15min":0.14599999999999999},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.22600000000000001},{"interface":"front","average":{"1min":0.249,"5min":0.24099999999999999,"15min":0.245},"min":{"1min":0.185,"5min":0.17599999999999999,"15min":0.16700000000000001},"max":{"1min":0.40899999999999997,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.23899999999999999}]},{"osd":1,"last update":"Fri Oct 6 00:16:02 2023","interfaces":[{"interface":"back","average":{"1min":0.26100000000000001,"5min":0.253,"15min":0.24399999999999999},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.153},"max":{"1min":0.39800000000000002,"5min":0.45400000000000001,"15min":0.48699999999999999},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.29499999999999998,"5min":0.27600000000000002,"15min":0.27200000000000002},"min":{"1min":0.23100000000000001,"5min":0.19600000000000001,"15min":0.16300000000000001},"max":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.51200000000000001},"last":0.27900000000000003}]}]},{"osd":0,"up_from":8,"seq":34359740665,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:16:06 2023","interfaces":[{"interface":"back","average":{"1min":0.215,"5min":0.19800000000000001,"15min":0.20000000000000001},"min":{"1min":0.157,"5min":0.128,"15min":0.128},"max":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.41299999999999998},"last":0.20000000000000001},{"interface":"front","average":{"1min":0.28399999999999997,"5min":0.27300000000000002,"15min":0.26400000000000001},"min":{"1min":0.214,"5min":0.189,"15min":0.152},"max":{"1min":0.443,"5min":0.443,"15min":0.47899999999999998},"last":0.26900000000000002}]},{"osd":2,"last update":"Fri Oct 6 00:16:06 2023","interfaces":[{"interface":"back","average":{"1min":0.25,"5min":0.23999999999999999,"15min":0.23200000000000001},"min":{"1min":0.183,"5min":0.16300000000000001,"15min":0.16300000000000001},"max":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.439},"last":0.23000000000000001},{"interface":"front","average":{"1min":0.313,"5min":0.30199999999999999,"15min":0.29599999999999999},"min":{"1min":0.249,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.53600000000000003},"last":0.30099999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740660,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.23899999999999999,"5min":0.218,"15min":0.20899999999999999},"min":{"1min":0.152,"5min":0.13100000000000001,"15min":0.13},"max":{"1min":0.41399999999999998,"5min":0.497,"15min":0.497},"last":0.158},{"interface":"front","average":{"1min":0.28000000000000003,"5min":0.26800000000000002,"15min":0.27300000000000002},"min":{"1min":0.16300000000000001,"5min":0.154,"15min":0.154},"max":{"1min":0.47599999999999998,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.248}]},{"osd":2,"last update":"Fri Oct 6 00:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.252,"5min":0.23699999999999999,"15min":0.23499999999999999},"min":{"1min":0.17599999999999999,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":0.437,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.20000000000000001},{"interface":"front","average":{"1min":0.28100000000000003,"5min":0.27400000000000002,"15min":0.27500000000000002},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.19900000000000001},"max":{"1min":0.50900000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.23100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:17:02.685 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:17:02.685 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:17:02.685 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:17:02.685 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:17:02.686 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:17:02.686 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:17:02.686 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:17:02.686 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:17:02.687 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:17:02.687 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:17:02.737 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:02.735+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:03.246 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:03.245+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:03.760 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:03.758+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:04.293 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:04.292+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:04.766 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:04.765+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:05.290 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:05.290+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:05.754 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:05.753+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:06.311 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:06.310+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:06.778 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:06.777+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:07.358 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:07.357+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:07.767 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:07.767+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:08.399 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:08.399+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:08.773 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:08.772+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:09.379 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:09.378+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:09.770 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:09.769+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:10.336 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:10.335+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:10.751 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:10.751+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:11.303 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:11.302+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:11.709 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:11.708+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:12.342 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:12.341+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:12.721 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:12.721+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:13.384 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:13.383+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:13.677 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:13.674+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:14.371 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:14.371+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:14.629 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:14.628+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:15.421 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:15.421+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:15.630 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:15.629+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:16.440 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:16.439+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:16.581 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:16.581+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:17.453 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:17.453+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:17.579 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:17.579+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:18.475 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:18.474+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:18.621 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:18.621+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:19.504 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:19.503+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:19.665 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:19.665+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:20.459 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:20.458+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:20.620 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:20.620+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:21.482 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:21.482+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:21.647 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:21.647+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:22.443 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:22.443+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:22.641 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:22.641+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:22.688 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:17:23.045 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:17:23.047 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:17:23.059 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":5884,"stamp":"2023-10-06T00:17:22.120745+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001698"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":1942,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:17:13.383498+0000","last_change":"2023-10-06T00:17:13.383498+0000","last_active":"2023-10-06T00:17:13.383498+0000","last_peered":"2023-10-06T00:17:13.383498+0000","last_clean":"2023-10-06T00:17:13.383498+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:17:13.383498+0000","last_undegraded":"2023-10-06T00:17:13.383498+0000","last_fullsized":"2023-10-06T00:17:13.383498+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1845,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:17:11.302462+0000","last_change":"2023-10-06T00:17:11.302462+0000","last_active":"2023-10-06T00:17:11.302462+0000","last_peered":"2023-10-06T00:17:11.302462+0000","last_clean":"2023-10-06T00:17:11.302462+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:17:11.302462+0000","last_undegraded":"2023-10-06T00:17:11.302462+0000","last_fullsized":"2023-10-06T00:17:11.302462+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8616,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:16:00.339983+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:16:00.339983+0000","last_peered":"2023-10-06T00:16:00.339983+0000","last_clean":"2023-10-06T00:16:00.339983+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:16:00.339983+0000","last_undegraded":"2023-10-06T00:16:00.339983+0000","last_fullsized":"2023-10-06T00:16:00.339983+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11097,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:17:18.474703+0000","last_change":"2023-10-06T00:17:18.474703+0000","last_active":"2023-10-06T00:17:18.474703+0000","last_peered":"2023-10-06T00:17:18.474703+0000","last_clean":"2023-10-06T00:17:18.474703+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:17:18.474703+0000","last_undegraded":"2023-10-06T00:17:18.474703+0000","last_fullsized":"2023-10-06T00:17:18.474703+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2361,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:17:08.399195+0000","last_change":"2023-10-06T00:17:08.399195+0000","last_active":"2023-10-06T00:17:08.399195+0000","last_peered":"2023-10-06T00:17:08.399195+0000","last_clean":"2023-10-06T00:17:08.399195+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:17:08.399195+0000","last_undegraded":"2023-10-06T00:17:08.399195+0000","last_fullsized":"2023-10-06T00:17:08.399195+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11745,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:17:17.453274+0000","last_change":"2023-10-06T00:17:17.453274+0000","last_active":"2023-10-06T00:17:17.453274+0000","last_peered":"2023-10-06T00:17:17.453274+0000","last_clean":"2023-10-06T00:17:17.453274+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:17:17.453274+0000","last_undegraded":"2023-10-06T00:17:17.453274+0000","last_fullsized":"2023-10-06T00:17:17.453274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11612,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-05T22:09:47.054310+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-05T22:09:47.054310+0000","last_peered":"2023-10-05T22:09:47.054310+0000","last_clean":"2023-10-05T22:09:47.054310+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-05T22:09:47.054310+0000","last_undegraded":"2023-10-05T22:09:47.054310+0000","last_fullsized":"2023-10-05T22:09:47.054310+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1782,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-05T22:09:47.054518+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-05T22:09:47.054518+0000","last_peered":"2023-10-05T22:09:47.054518+0000","last_clean":"2023-10-05T22:09:47.054518+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-05T22:09:47.054518+0000","last_undegraded":"2023-10-05T22:09:47.054518+0000","last_fullsized":"2023-10-05T22:09:47.054518+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1766,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:17:07.357873+0000","last_change":"2023-10-06T00:17:07.357873+0000","last_active":"2023-10-06T00:17:07.357873+0000","last_peered":"2023-10-06T00:17:07.357873+0000","last_clean":"2023-10-06T00:17:07.357873+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:17:07.357873+0000","last_undegraded":"2023-10-06T00:17:07.357873+0000","last_fullsized":"2023-10-06T00:17:07.357873+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740668,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:17:02 2023","interfaces":[{"interface":"back","average":{"1min":0.214,"5min":0.22600000000000001,"15min":0.223},"min":{"1min":0.156,"5min":0.155,"15min":0.151},"max":{"1min":0.32400000000000001,"5min":1.153,"15min":1.153},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.23200000000000001,"5min":0.23599999999999999,"15min":0.24399999999999999},"min":{"1min":0.17699999999999999,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.34799999999999998,"5min":0.41899999999999998,"15min":0.48399999999999999},"last":0.22800000000000001}]},{"osd":1,"last update":"Fri Oct 6 00:17:02 2023","interfaces":[{"interface":"back","average":{"1min":0.25700000000000001,"5min":0.252,"15min":0.246},"min":{"1min":0.16800000000000001,"5min":0.16800000000000001,"15min":0.16800000000000001},"max":{"1min":0.39500000000000002,"5min":0.41499999999999998,"15min":0.48699999999999999},"last":0.21299999999999999},{"interface":"front","average":{"1min":0.27700000000000002,"5min":0.27400000000000002,"15min":0.27300000000000002},"min":{"1min":0.219,"5min":0.19600000000000001,"15min":0.16300000000000001},"max":{"1min":0.40699999999999997,"5min":0.44900000000000001,"15min":0.51200000000000001},"last":0.28199999999999997}]}]},{"osd":0,"up_from":8,"seq":34359740669,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20100000000000001},"min":{"1min":0.14399999999999999,"5min":0.13600000000000001,"15min":0.128},"max":{"1min":0.27900000000000003,"5min":0.34999999999999998,"15min":0.41299999999999998},"last":0.192},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.27300000000000002,"15min":0.26500000000000001},"min":{"1min":0.186,"5min":0.186,"15min":0.152},"max":{"1min":0.35199999999999998,"5min":0.443,"15min":0.47899999999999998},"last":0.26500000000000001}]},{"osd":2,"last update":"Fri Oct 6 00:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.23699999999999999,"5min":0.24099999999999999,"15min":0.23400000000000001},"min":{"1min":0.17100000000000001,"5min":0.16700000000000001,"15min":0.16300000000000001},"max":{"1min":0.32200000000000001,"5min":0.40699999999999997,"15min":0.439},"last":0.22600000000000001},{"interface":"front","average":{"1min":0.29599999999999999,"5min":0.30199999999999999,"15min":0.29699999999999999},"min":{"1min":0.215,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.38100000000000001,"5min":0.47499999999999998,"15min":0.53600000000000003},"last":0.30199999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740664,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.23899999999999999,"5min":0.218,"15min":0.20899999999999999},"min":{"1min":0.152,"5min":0.13100000000000001,"15min":0.13},"max":{"1min":0.41399999999999998,"5min":0.497,"15min":0.497},"last":0.19800000000000001},{"interface":"front","average":{"1min":0.28000000000000003,"5min":0.26800000000000002,"15min":0.27300000000000002},"min":{"1min":0.16300000000000001,"5min":0.154,"15min":0.154},"max":{"1min":0.47599999999999998,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.24399999999999999}]},{"osd":2,"last update":"Fri Oct 6 00:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.252,"5min":0.23699999999999999,"15min":0.23499999999999999},"min":{"1min":0.17599999999999999,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":0.437,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.28100000000000003,"5min":0.27400000000000002,"15min":0.27500000000000002},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.19900000000000001},"max":{"1min":0.50900000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.23300000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:17:23.061 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:17:23.061 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:17:23.061 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:17:23.061 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:17:23.062 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:17:23.062 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:17:23.062 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:17:23.062 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:17:23.063 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:17:23.063 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:17:23.436 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:23.435+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:23.603 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:23.603+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:24.460 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:24.460+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:24.622 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:24.621+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:25.477 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:25.476+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:25.617 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:25.615+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:26.454 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:26.453+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:26.627 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:26.626+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:27.443 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:27.441+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:27.673 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:27.672+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:28.408 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:28.406+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:28.701 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:28.699+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:29.429 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:29.428+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:29.727 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:29.726+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:30.435 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:30.433+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:30.708 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:30.707+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:31.405 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:31.404+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:31.718 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:31.716+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:32.439 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:32.438+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:32.719 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:32.717+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:33.412 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:33.411+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:33.753 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:33.751+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:34.409 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:34.408+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:34.772 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:34.771+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:35.431 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:35.430+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:35.815 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:35.813+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:36.408 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:36.406+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:36.793 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:36.792+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:37.448 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:37.446+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:37.773 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:37.772+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:38.481 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:38.480+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:38.793 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:38.792+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:39.450 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:39.448+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:39.776 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:39.774+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:40.406 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:40.404+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:40.793 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:40.791+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:41.447 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:41.445+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:41.792 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:41.790+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:42.495 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:42.494+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:42.774 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:42.772+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:43.064 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:17:43.395 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:17:43.397 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:17:43.408 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":5894,"stamp":"2023-10-06T00:17:42.123914+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001977"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":1945,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:17:31.405231+0000","last_change":"2023-10-06T00:17:31.405231+0000","last_active":"2023-10-06T00:17:31.405231+0000","last_peered":"2023-10-06T00:17:31.405231+0000","last_clean":"2023-10-06T00:17:31.405231+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:17:31.405231+0000","last_undegraded":"2023-10-06T00:17:31.405231+0000","last_fullsized":"2023-10-06T00:17:31.405231+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1848,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:17:27.442654+0000","last_change":"2023-10-06T00:17:27.442654+0000","last_active":"2023-10-06T00:17:27.442654+0000","last_peered":"2023-10-06T00:17:27.442654+0000","last_clean":"2023-10-06T00:17:27.442654+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:17:27.442654+0000","last_undegraded":"2023-10-06T00:17:27.442654+0000","last_fullsized":"2023-10-06T00:17:27.442654+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8616,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:16:00.339983+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:16:00.339983+0000","last_peered":"2023-10-06T00:16:00.339983+0000","last_clean":"2023-10-06T00:16:00.339983+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:16:00.339983+0000","last_undegraded":"2023-10-06T00:16:00.339983+0000","last_fullsized":"2023-10-06T00:16:00.339983+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11100,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:17:38.480967+0000","last_change":"2023-10-06T00:17:38.480967+0000","last_active":"2023-10-06T00:17:38.480967+0000","last_peered":"2023-10-06T00:17:38.480967+0000","last_clean":"2023-10-06T00:17:38.480967+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:17:38.480967+0000","last_undegraded":"2023-10-06T00:17:38.480967+0000","last_fullsized":"2023-10-06T00:17:38.480967+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2364,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:17:26.454197+0000","last_change":"2023-10-06T00:17:26.454197+0000","last_active":"2023-10-06T00:17:26.454197+0000","last_peered":"2023-10-06T00:17:26.454197+0000","last_clean":"2023-10-06T00:17:26.454197+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:17:26.454197+0000","last_undegraded":"2023-10-06T00:17:26.454197+0000","last_fullsized":"2023-10-06T00:17:26.454197+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11748,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:17:36.407798+0000","last_change":"2023-10-06T00:17:36.407798+0000","last_active":"2023-10-06T00:17:36.407798+0000","last_peered":"2023-10-06T00:17:36.407798+0000","last_clean":"2023-10-06T00:17:36.407798+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:17:36.407798+0000","last_undegraded":"2023-10-06T00:17:36.407798+0000","last_fullsized":"2023-10-06T00:17:36.407798+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11612,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-05T22:09:47.054310+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-05T22:09:47.054310+0000","last_peered":"2023-10-05T22:09:47.054310+0000","last_clean":"2023-10-05T22:09:47.054310+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-05T22:09:47.054310+0000","last_undegraded":"2023-10-05T22:09:47.054310+0000","last_fullsized":"2023-10-05T22:09:47.054310+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1782,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-05T22:09:47.054518+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-05T22:09:47.054518+0000","last_peered":"2023-10-05T22:09:47.054518+0000","last_clean":"2023-10-05T22:09:47.054518+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-05T22:09:47.054518+0000","last_undegraded":"2023-10-05T22:09:47.054518+0000","last_fullsized":"2023-10-05T22:09:47.054518+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1769,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:17:23.435938+0000","last_change":"2023-10-06T00:17:23.435938+0000","last_active":"2023-10-06T00:17:23.435938+0000","last_peered":"2023-10-06T00:17:23.435938+0000","last_clean":"2023-10-06T00:17:23.435938+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:17:23.435938+0000","last_undegraded":"2023-10-06T00:17:23.435938+0000","last_fullsized":"2023-10-06T00:17:23.435938+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740672,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:17:02 2023","interfaces":[{"interface":"back","average":{"1min":0.214,"5min":0.22600000000000001,"15min":0.223},"min":{"1min":0.156,"5min":0.155,"15min":0.151},"max":{"1min":0.32400000000000001,"5min":1.153,"15min":1.153},"last":0.161},{"interface":"front","average":{"1min":0.23200000000000001,"5min":0.23599999999999999,"15min":0.24399999999999999},"min":{"1min":0.17699999999999999,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.34799999999999998,"5min":0.41899999999999998,"15min":0.48399999999999999},"last":0.20300000000000001}]},{"osd":1,"last update":"Fri Oct 6 00:17:02 2023","interfaces":[{"interface":"back","average":{"1min":0.25700000000000001,"5min":0.252,"15min":0.246},"min":{"1min":0.16800000000000001,"5min":0.16800000000000001,"15min":0.16800000000000001},"max":{"1min":0.39500000000000002,"5min":0.41499999999999998,"15min":0.48699999999999999},"last":0.191},{"interface":"front","average":{"1min":0.27700000000000002,"5min":0.27400000000000002,"15min":0.27300000000000002},"min":{"1min":0.219,"5min":0.19600000000000001,"15min":0.16300000000000001},"max":{"1min":0.40699999999999997,"5min":0.44900000000000001,"15min":0.51200000000000001},"last":0.24099999999999999}]}]},{"osd":0,"up_from":8,"seq":34359740673,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20100000000000001},"min":{"1min":0.14399999999999999,"5min":0.13600000000000001,"15min":0.128},"max":{"1min":0.27900000000000003,"5min":0.34999999999999998,"15min":0.41299999999999998},"last":0.20200000000000001},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.27300000000000002,"15min":0.26500000000000001},"min":{"1min":0.186,"5min":0.186,"15min":0.152},"max":{"1min":0.35199999999999998,"5min":0.443,"15min":0.47899999999999998},"last":0.22}]},{"osd":2,"last update":"Fri Oct 6 00:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.23699999999999999,"5min":0.24099999999999999,"15min":0.23400000000000001},"min":{"1min":0.17100000000000001,"5min":0.16700000000000001,"15min":0.16300000000000001},"max":{"1min":0.32200000000000001,"5min":0.40699999999999997,"15min":0.439},"last":0.185},{"interface":"front","average":{"1min":0.29599999999999999,"5min":0.30199999999999999,"15min":0.29699999999999999},"min":{"1min":0.215,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.38100000000000001,"5min":0.47499999999999998,"15min":0.53600000000000003},"last":0.252}]}]},{"osd":1,"up_from":8,"seq":34359740668,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:17:34 2023","interfaces":[{"interface":"back","average":{"1min":0.20699999999999999,"5min":0.221,"15min":0.20799999999999999},"min":{"1min":0.14199999999999999,"5min":0.13100000000000001,"15min":0.13},"max":{"1min":0.25600000000000001,"5min":0.497,"15min":0.497},"last":0.16400000000000001},{"interface":"front","average":{"1min":0.23799999999999999,"5min":0.26200000000000001,"15min":0.26900000000000002},"min":{"1min":0.187,"5min":0.154,"15min":0.154},"max":{"1min":0.30299999999999999,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.251}]},{"osd":2,"last update":"Fri Oct 6 00:17:34 2023","interfaces":[{"interface":"back","average":{"1min":0.23300000000000001,"5min":0.23999999999999999,"15min":0.23400000000000001},"min":{"1min":0.184,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":0.27400000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.216},{"interface":"front","average":{"1min":0.23100000000000001,"5min":0.26900000000000002,"15min":0.27100000000000002},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.27400000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.26700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:17:43.410 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:17:43.410 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:17:43.410 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:17:43.410 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:17:43.411 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:17:43.411 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:17:43.411 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:17:43.411 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:17:43.411 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:17:43.411 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:17:43.473 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:43.471+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:43.745 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:43.743+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:44.485 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:44.484+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:44.723 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:44.722+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:45.453 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:45.452+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:45.705 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:45.704+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:46.484 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:46.482+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:46.740 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:46.738+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:47.479 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:47.477+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:47.697 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:47.696+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:48.497 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:48.496+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:48.681 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:48.679+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:49.489 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:49.487+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:49.680 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:49.678+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:50.465 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:50.464+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:50.719 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:50.717+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:51.446 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:51.444+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:51.760 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:51.758+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:52.488 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:52.486+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:52.717 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:52.716+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:53.482 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:53.480+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:53.684 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:53.682+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:54.511 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:54.509+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:54.649 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:54.647+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:55.521 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:55.519+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:55.624 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:55.622+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:56.523 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:56.521+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:56.577 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:56.575+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:57.547 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:57.545+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:57.568 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:57.567+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:58.526 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:58.524+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:58.555 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:58.554+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:59.487 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:17:59.486+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:17:59.543 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:17:59.542+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:00.485 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:00.484+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:00.542 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:00.540+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:01.466 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:01.465+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:01.533 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:01.532+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:02.515 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:02.513+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:02.538 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:02.537+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:03.412 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:18:03.516 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:03.516+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:03.553 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:03.552+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:03.740 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:18:03.742 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:18:03.753 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":5904,"stamp":"2023-10-06T00:18:02.127528+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002276"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":1948,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:17:52.487880+0000","last_change":"2023-10-06T00:17:52.487880+0000","last_active":"2023-10-06T00:17:52.487880+0000","last_peered":"2023-10-06T00:17:52.487880+0000","last_clean":"2023-10-06T00:17:52.487880+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:17:52.487880+0000","last_undegraded":"2023-10-06T00:17:52.487880+0000","last_fullsized":"2023-10-06T00:17:52.487880+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1851,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:17:49.488462+0000","last_change":"2023-10-06T00:17:49.488462+0000","last_active":"2023-10-06T00:17:49.488462+0000","last_peered":"2023-10-06T00:17:49.488462+0000","last_clean":"2023-10-06T00:17:49.488462+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:17:49.488462+0000","last_undegraded":"2023-10-06T00:17:49.488462+0000","last_fullsized":"2023-10-06T00:17:49.488462+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8616,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:16:00.339983+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:16:00.339983+0000","last_peered":"2023-10-06T00:16:00.339983+0000","last_clean":"2023-10-06T00:16:00.339983+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:16:00.339983+0000","last_undegraded":"2023-10-06T00:16:00.339983+0000","last_fullsized":"2023-10-06T00:16:00.339983+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11103,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:17:57.546848+0000","last_change":"2023-10-06T00:17:57.546848+0000","last_active":"2023-10-06T00:17:57.546848+0000","last_peered":"2023-10-06T00:17:57.546848+0000","last_clean":"2023-10-06T00:17:57.546848+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:17:57.546848+0000","last_undegraded":"2023-10-06T00:17:57.546848+0000","last_fullsized":"2023-10-06T00:17:57.546848+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2367,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:17:43.473055+0000","last_change":"2023-10-06T00:17:43.473055+0000","last_active":"2023-10-06T00:17:43.473055+0000","last_peered":"2023-10-06T00:17:43.473055+0000","last_clean":"2023-10-06T00:17:43.473055+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:17:43.473055+0000","last_undegraded":"2023-10-06T00:17:43.473055+0000","last_fullsized":"2023-10-06T00:17:43.473055+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11751,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:17:55.520717+0000","last_change":"2023-10-06T00:17:55.520717+0000","last_active":"2023-10-06T00:17:55.520717+0000","last_peered":"2023-10-06T00:17:55.520717+0000","last_clean":"2023-10-06T00:17:55.520717+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:17:55.520717+0000","last_undegraded":"2023-10-06T00:17:55.520717+0000","last_fullsized":"2023-10-06T00:17:55.520717+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11612,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-05T22:09:47.054310+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-05T22:09:47.054310+0000","last_peered":"2023-10-05T22:09:47.054310+0000","last_clean":"2023-10-05T22:09:47.054310+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-05T22:09:47.054310+0000","last_undegraded":"2023-10-05T22:09:47.054310+0000","last_fullsized":"2023-10-05T22:09:47.054310+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1782,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-05T22:09:47.054518+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-05T22:09:47.054518+0000","last_peered":"2023-10-05T22:09:47.054518+0000","last_clean":"2023-10-05T22:09:47.054518+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-05T22:09:47.054518+0000","last_undegraded":"2023-10-05T22:09:47.054518+0000","last_fullsized":"2023-10-05T22:09:47.054518+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1772,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:17:41.446462+0000","last_change":"2023-10-06T00:17:41.446462+0000","last_active":"2023-10-06T00:17:41.446462+0000","last_peered":"2023-10-06T00:17:41.446462+0000","last_clean":"2023-10-06T00:17:41.446462+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:17:41.446462+0000","last_undegraded":"2023-10-06T00:17:41.446462+0000","last_fullsized":"2023-10-06T00:17:41.446462+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740676,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:17:02 2023","interfaces":[{"interface":"back","average":{"1min":0.214,"5min":0.22600000000000001,"15min":0.223},"min":{"1min":0.156,"5min":0.155,"15min":0.151},"max":{"1min":0.32400000000000001,"5min":1.153,"15min":1.153},"last":0.20399999999999999},{"interface":"front","average":{"1min":0.23200000000000001,"5min":0.23599999999999999,"15min":0.24399999999999999},"min":{"1min":0.17699999999999999,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.34799999999999998,"5min":0.41899999999999998,"15min":0.48399999999999999},"last":0.221}]},{"osd":1,"last update":"Fri Oct 6 00:17:02 2023","interfaces":[{"interface":"back","average":{"1min":0.25700000000000001,"5min":0.252,"15min":0.246},"min":{"1min":0.16800000000000001,"5min":0.16800000000000001,"15min":0.16800000000000001},"max":{"1min":0.39500000000000002,"5min":0.41499999999999998,"15min":0.48699999999999999},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.27700000000000002,"5min":0.27400000000000002,"15min":0.27300000000000002},"min":{"1min":0.219,"5min":0.19600000000000001,"15min":0.16300000000000001},"max":{"1min":0.40699999999999997,"5min":0.44900000000000001,"15min":0.51200000000000001},"last":0.251}]}]},{"osd":0,"up_from":8,"seq":34359740677,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20100000000000001},"min":{"1min":0.14399999999999999,"5min":0.13600000000000001,"15min":0.128},"max":{"1min":0.27900000000000003,"5min":0.34999999999999998,"15min":0.41299999999999998},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.27300000000000002,"15min":0.26500000000000001},"min":{"1min":0.186,"5min":0.186,"15min":0.152},"max":{"1min":0.35199999999999998,"5min":0.443,"15min":0.47899999999999998},"last":0.38}]},{"osd":2,"last update":"Fri Oct 6 00:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.23699999999999999,"5min":0.24099999999999999,"15min":0.23400000000000001},"min":{"1min":0.17100000000000001,"5min":0.16700000000000001,"15min":0.16300000000000001},"max":{"1min":0.32200000000000001,"5min":0.40699999999999997,"15min":0.439},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.29599999999999999,"5min":0.30199999999999999,"15min":0.29699999999999999},"min":{"1min":0.215,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.38100000000000001,"5min":0.47499999999999998,"15min":0.53600000000000003},"last":0.40999999999999998}]}]},{"osd":1,"up_from":8,"seq":34359740672,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:17:34 2023","interfaces":[{"interface":"back","average":{"1min":0.20699999999999999,"5min":0.221,"15min":0.20799999999999999},"min":{"1min":0.14199999999999999,"5min":0.13100000000000001,"15min":0.13},"max":{"1min":0.25600000000000001,"5min":0.497,"15min":0.497},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.23799999999999999,"5min":0.26200000000000001,"15min":0.26900000000000002},"min":{"1min":0.187,"5min":0.154,"15min":0.154},"max":{"1min":0.30299999999999999,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.27600000000000002}]},{"osd":2,"last update":"Fri Oct 6 00:17:34 2023","interfaces":[{"interface":"back","average":{"1min":0.23300000000000001,"5min":0.23999999999999999,"15min":0.23400000000000001},"min":{"1min":0.184,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":0.27400000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.248},{"interface":"front","average":{"1min":0.23100000000000001,"5min":0.26900000000000002,"15min":0.27100000000000002},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.27400000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.22800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:18:03.754 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:18:03.755 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:18:03.755 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:18:03.755 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:18:03.755 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:18:03.755 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:18:03.756 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:18:03.756 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:18:03.756 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:18:03.756 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:18:04.518 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:04.517+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:04.526 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:04.526+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:05.480 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:05.480+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:05.519 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:05.518+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:06.442 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:06.441+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:06.473 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:06.473+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:07.451 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:07.451+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:07.488 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:07.488+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:08.412 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:08.412+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:08.440 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:08.440+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:09.433 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:09.431+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:09.463 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:09.462+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:10.472 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:10.472+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:10.489 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:10.489+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:11.460 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:11.460+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:11.534 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:11.534+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:12.429 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:12.428+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:12.554 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:12.553+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:13.417 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:13.417+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:13.595 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:13.594+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:14.406 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:14.406+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:14.576 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:14.575+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:15.411 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:15.411+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:15.549 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:15.549+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:16.429 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:16.428+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:16.516 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:16.516+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:17.404 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:17.404+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:17.487 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:17.487+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:18.359 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:18.358+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:18.500 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:18.499+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:19.310 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:19.310+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:19.496 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:19.496+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:20.316 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:20.315+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:20.509 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:20.508+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:21.299 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:21.298+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:21.468 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:21.468+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:22.269 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:22.267+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:22.438 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:22.438+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:23.232 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:23.230+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:23.445 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:23.444+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:23.757 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:18:24.094 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:18:24.096 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:18:24.106 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":5914,"stamp":"2023-10-06T00:18:22.130700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002015"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":1951,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:18:08.440299+0000","last_change":"2023-10-06T00:18:08.440299+0000","last_active":"2023-10-06T00:18:08.440299+0000","last_peered":"2023-10-06T00:18:08.440299+0000","last_clean":"2023-10-06T00:18:08.440299+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:18:08.440299+0000","last_undegraded":"2023-10-06T00:18:08.440299+0000","last_fullsized":"2023-10-06T00:18:08.440299+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1854,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:18:07.488217+0000","last_change":"2023-10-06T00:18:07.488217+0000","last_active":"2023-10-06T00:18:07.488217+0000","last_peered":"2023-10-06T00:18:07.488217+0000","last_clean":"2023-10-06T00:18:07.488217+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:18:07.488217+0000","last_undegraded":"2023-10-06T00:18:07.488217+0000","last_fullsized":"2023-10-06T00:18:07.488217+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8616,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:16:00.339983+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:16:00.339983+0000","last_peered":"2023-10-06T00:16:00.339983+0000","last_clean":"2023-10-06T00:16:00.339983+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:16:00.339983+0000","last_undegraded":"2023-10-06T00:16:00.339983+0000","last_fullsized":"2023-10-06T00:16:00.339983+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11106,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:18:13.594576+0000","last_change":"2023-10-06T00:18:13.594576+0000","last_active":"2023-10-06T00:18:13.594576+0000","last_peered":"2023-10-06T00:18:13.594576+0000","last_clean":"2023-10-06T00:18:13.594576+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:18:13.594576+0000","last_undegraded":"2023-10-06T00:18:13.594576+0000","last_fullsized":"2023-10-06T00:18:13.594576+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2370,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:18:05.479902+0000","last_change":"2023-10-06T00:18:05.479902+0000","last_active":"2023-10-06T00:18:05.479902+0000","last_peered":"2023-10-06T00:18:05.479902+0000","last_clean":"2023-10-06T00:18:05.479902+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:18:05.479902+0000","last_undegraded":"2023-10-06T00:18:05.479902+0000","last_fullsized":"2023-10-06T00:18:05.479902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11754,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:18:10.489401+0000","last_change":"2023-10-06T00:18:10.489401+0000","last_active":"2023-10-06T00:18:10.489401+0000","last_peered":"2023-10-06T00:18:10.489401+0000","last_clean":"2023-10-06T00:18:10.489401+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:18:10.489401+0000","last_undegraded":"2023-10-06T00:18:10.489401+0000","last_fullsized":"2023-10-06T00:18:10.489401+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11612,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-05T22:09:47.054310+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-05T22:09:47.054310+0000","last_peered":"2023-10-05T22:09:47.054310+0000","last_clean":"2023-10-05T22:09:47.054310+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-05T22:09:47.054310+0000","last_undegraded":"2023-10-05T22:09:47.054310+0000","last_fullsized":"2023-10-05T22:09:47.054310+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1782,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-05T22:09:47.054518+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-05T22:09:47.054518+0000","last_peered":"2023-10-05T22:09:47.054518+0000","last_clean":"2023-10-05T22:09:47.054518+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-05T22:09:47.054518+0000","last_undegraded":"2023-10-05T22:09:47.054518+0000","last_fullsized":"2023-10-05T22:09:47.054518+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1778,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:18:15.549135+0000","last_change":"2023-10-06T00:18:15.549135+0000","last_active":"2023-10-06T00:18:15.549135+0000","last_peered":"2023-10-06T00:18:15.549135+0000","last_clean":"2023-10-06T00:18:15.549135+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:18:15.549135+0000","last_undegraded":"2023-10-06T00:18:15.549135+0000","last_fullsized":"2023-10-06T00:18:15.549135+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740680,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:18:05 2023","interfaces":[{"interface":"back","average":{"1min":0.23200000000000001,"5min":0.23000000000000001,"15min":0.222},"min":{"1min":0.161,"5min":0.155,"15min":0.151},"max":{"1min":0.379,"5min":1.153,"15min":1.153},"last":0.182},{"interface":"front","average":{"1min":0.24099999999999999,"5min":0.23799999999999999,"15min":0.24299999999999999},"min":{"1min":0.17299999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.40100000000000002,"5min":0.41899999999999998,"15min":0.48399999999999999},"last":0.22600000000000001}]},{"osd":1,"last update":"Fri Oct 6 00:18:05 2023","interfaces":[{"interface":"back","average":{"1min":0.26300000000000001,"5min":0.255,"15min":0.246},"min":{"1min":0.184,"5min":0.16800000000000001,"15min":0.16800000000000001},"max":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.48699999999999999},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.29099999999999998,"5min":0.28000000000000003,"15min":0.27400000000000002},"min":{"1min":0.24099999999999999,"5min":0.217,"15min":0.16300000000000001},"max":{"1min":0.433,"5min":0.44900000000000001,"15min":0.51200000000000001},"last":0.27400000000000002}]}]},{"osd":0,"up_from":8,"seq":34359740681,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.19800000000000001,"5min":0.20499999999999999,"15min":0.20100000000000001},"min":{"1min":0.14699999999999999,"5min":0.13600000000000001,"15min":0.128},"max":{"1min":0.28999999999999998,"5min":0.34999999999999998,"15min":0.41299999999999998},"last":0.17599999999999999},{"interface":"front","average":{"1min":0.26500000000000001,"5min":0.27300000000000002,"15min":0.26500000000000001},"min":{"1min":0.22,"5min":0.186,"15min":0.152},"max":{"1min":0.38,"5min":0.443,"15min":0.47899999999999998},"last":0.246}]},{"osd":2,"last update":"Fri Oct 6 00:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.22800000000000001,"5min":0.23899999999999999,"15min":0.23400000000000001},"min":{"1min":0.185,"5min":0.16700000000000001,"15min":0.16300000000000001},"max":{"1min":0.34899999999999998,"5min":0.40699999999999997,"15min":0.439},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.28999999999999998,"5min":0.29999999999999999,"15min":0.29699999999999999},"min":{"1min":0.252,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.40999999999999998,"5min":0.47499999999999998,"15min":0.53600000000000003},"last":0.30499999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740676,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:17:34 2023","interfaces":[{"interface":"back","average":{"1min":0.20699999999999999,"5min":0.221,"15min":0.20799999999999999},"min":{"1min":0.14199999999999999,"5min":0.13100000000000001,"15min":0.13},"max":{"1min":0.25600000000000001,"5min":0.497,"15min":0.497},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.23799999999999999,"5min":0.26200000000000001,"15min":0.26900000000000002},"min":{"1min":0.187,"5min":0.154,"15min":0.154},"max":{"1min":0.30299999999999999,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.27900000000000003}]},{"osd":2,"last update":"Fri Oct 6 00:17:34 2023","interfaces":[{"interface":"back","average":{"1min":0.23300000000000001,"5min":0.23999999999999999,"15min":0.23400000000000001},"min":{"1min":0.184,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":0.27400000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.23100000000000001,"5min":0.26900000000000002,"15min":0.27100000000000002},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.27400000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.25}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:18:24.108 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:18:24.108 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:18:24.108 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:18:24.109 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:18:24.109 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:18:24.109 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:18:24.109 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:18:24.109 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:18:24.110 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:18:24.110 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.7 2023-10-06T00:18:24.198 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:24.196+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:24.434 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:24.432+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:24.485 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 2.7 on osd.1 to deep-scrub 2023-10-06T00:18:24.497 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.6 2023-10-06T00:18:24.837 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 2.6 on osd.1 to deep-scrub 2023-10-06T00:18:24.849 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.1 2023-10-06T00:18:25.184 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:25.182+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:25.218 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 3.1 on osd.0 to deep-scrub 2023-10-06T00:18:25.230 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.2 2023-10-06T00:18:25.446 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:25.444+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:25.585 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 3.2 on osd.1 to deep-scrub 2023-10-06T00:18:25.595 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.3 2023-10-06T00:18:25.961 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 2.3 on osd.1 to deep-scrub 2023-10-06T00:18:25.973 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.0 2023-10-06T00:18:26.184 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:26.182+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:26.298 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 3.0 on osd.1 to deep-scrub 2023-10-06T00:18:26.309 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.3 2023-10-06T00:18:26.443 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:26.441+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:26.646 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 3.3 on osd.0 to deep-scrub 2023-10-06T00:18:26.657 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.2 2023-10-06T00:18:27.031 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 2.2 on osd.0 to deep-scrub 2023-10-06T00:18:27.043 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.4 2023-10-06T00:18:27.216 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:27.214+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:27.393 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 2.4 on osd.1 to deep-scrub 2023-10-06T00:18:27.404 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:18:27.420 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:27.418+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:28.171 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:28.169+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:28.390 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:28.388+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:29.149 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:29.148+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:29.350 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:29.348+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:30.135 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:30.134+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:30.334 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:30.332+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:31.175 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:31.173+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:31.380 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:31.378+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:32.198 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:32.196+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:32.357 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:32.355+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:33.174 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:33.172+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:33.366 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:33.364+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:34.151 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:34.149+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:34.339 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:34.337+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:35.129 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:35.127+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:35.362 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:35.360+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:36.120 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:36.118+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:36.389 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:36.387+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:37.116 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:37.114+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:37.354 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:37.352+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:38.077 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:38.075+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:38.398 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:38.397+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:39.088 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:39.086+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:39.382 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:39.380+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:40.053 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:40.051+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:40.354 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:40.352+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:41.039 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:41.037+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:41.332 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:41.331+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:42.010 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:42.009+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:42.288 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:42.287+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:42.993 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:42.991+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:43.262 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:43.260+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:43.992 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:43.991+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:44.225 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:44.223+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:44.956 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:44.955+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:45.214 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:45.213+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:45.916 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:45.914+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:46.170 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:46.169+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:46.912 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:46.911+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:47.168 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:47.166+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:47.405 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:18:47.745 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:18:47.747 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:18:47.757 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":5926,"stamp":"2023-10-06T00:18:46.134385+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001810"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":1958,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:18:36.389000+0000","last_change":"2023-10-06T00:18:36.389000+0000","last_active":"2023-10-06T00:18:36.389000+0000","last_peered":"2023-10-06T00:18:36.389000+0000","last_clean":"2023-10-06T00:18:36.389000+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:18:36.389000+0000","last_undegraded":"2023-10-06T00:18:36.389000+0000","last_fullsized":"2023-10-06T00:18:36.389000+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1864,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:18:43.261511+0000","last_change":"2023-10-06T00:18:43.261511+0000","last_active":"2023-10-06T00:18:43.261511+0000","last_peered":"2023-10-06T00:18:43.261511+0000","last_clean":"2023-10-06T00:18:43.261511+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:18:43.261511+0000","last_undegraded":"2023-10-06T00:18:43.261511+0000","last_fullsized":"2023-10-06T00:18:43.261511+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8617,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:16:00.340028+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:16:00.340028+0000","last_peered":"2023-10-06T00:16:00.340028+0000","last_clean":"2023-10-06T00:16:00.340028+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:16:00.340028+0000","last_undegraded":"2023-10-06T00:16:00.340028+0000","last_fullsized":"2023-10-06T00:16:00.340028+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 144s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11113,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:18:39.381335+0000","last_change":"2023-10-06T00:18:39.381335+0000","last_active":"2023-10-06T00:18:39.381335+0000","last_peered":"2023-10-06T00:18:39.381335+0000","last_clean":"2023-10-06T00:18:39.381335+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:18:39.381335+0000","last_undegraded":"2023-10-06T00:18:39.381335+0000","last_fullsized":"2023-10-06T00:18:39.381335+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2380,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:18:41.332308+0000","last_change":"2023-10-06T00:18:41.332308+0000","last_active":"2023-10-06T00:18:41.332308+0000","last_peered":"2023-10-06T00:18:41.332308+0000","last_clean":"2023-10-06T00:18:41.332308+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:18:41.332308+0000","last_undegraded":"2023-10-06T00:18:41.332308+0000","last_fullsized":"2023-10-06T00:18:41.332308+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11761,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:18:38.398067+0000","last_change":"2023-10-06T00:18:38.398067+0000","last_active":"2023-10-06T00:18:38.398067+0000","last_peered":"2023-10-06T00:18:38.398067+0000","last_clean":"2023-10-06T00:18:38.398067+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:18:38.398067+0000","last_undegraded":"2023-10-06T00:18:38.398067+0000","last_fullsized":"2023-10-06T00:18:38.398067+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11613,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:16:00.896651+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-06T00:16:00.896651+0000","last_peered":"2023-10-06T00:16:00.896651+0000","last_clean":"2023-10-06T00:16:00.896651+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-06T00:16:00.896651+0000","last_undegraded":"2023-10-06T00:16:00.896651+0000","last_fullsized":"2023-10-06T00:16:00.896651+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1783,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:16:01.247598+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-06T00:16:01.247598+0000","last_peered":"2023-10-06T00:16:01.247598+0000","last_clean":"2023-10-06T00:16:01.247598+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-06T00:16:01.247598+0000","last_undegraded":"2023-10-06T00:16:01.247598+0000","last_fullsized":"2023-10-06T00:16:01.247598+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1785,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:18:40.353623+0000","last_change":"2023-10-06T00:18:40.353623+0000","last_active":"2023-10-06T00:18:40.353623+0000","last_peered":"2023-10-06T00:18:40.353623+0000","last_clean":"2023-10-06T00:18:40.353623+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:18:40.353623+0000","last_undegraded":"2023-10-06T00:18:40.353623+0000","last_fullsized":"2023-10-06T00:18:40.353623+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740685,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:18:05 2023","interfaces":[{"interface":"back","average":{"1min":0.23200000000000001,"5min":0.23000000000000001,"15min":0.222},"min":{"1min":0.161,"5min":0.155,"15min":0.151},"max":{"1min":0.379,"5min":1.153,"15min":1.153},"last":0.191},{"interface":"front","average":{"1min":0.24099999999999999,"5min":0.23799999999999999,"15min":0.24299999999999999},"min":{"1min":0.17299999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.40100000000000002,"5min":0.41899999999999998,"15min":0.48399999999999999},"last":0.21299999999999999}]},{"osd":1,"last update":"Fri Oct 6 00:18:05 2023","interfaces":[{"interface":"back","average":{"1min":0.26300000000000001,"5min":0.255,"15min":0.246},"min":{"1min":0.184,"5min":0.16800000000000001,"15min":0.16800000000000001},"max":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.48699999999999999},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.29099999999999998,"5min":0.28000000000000003,"15min":0.27400000000000002},"min":{"1min":0.24099999999999999,"5min":0.217,"15min":0.16300000000000001},"max":{"1min":0.433,"5min":0.44900000000000001,"15min":0.51200000000000001},"last":0.27000000000000002}]}]},{"osd":0,"up_from":8,"seq":34359740686,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.19800000000000001,"5min":0.20499999999999999,"15min":0.20100000000000001},"min":{"1min":0.14699999999999999,"5min":0.13600000000000001,"15min":0.128},"max":{"1min":0.28999999999999998,"5min":0.34999999999999998,"15min":0.41299999999999998},"last":0.20200000000000001},{"interface":"front","average":{"1min":0.26500000000000001,"5min":0.27300000000000002,"15min":0.26500000000000001},"min":{"1min":0.22,"5min":0.186,"15min":0.152},"max":{"1min":0.38,"5min":0.443,"15min":0.47899999999999998},"last":0.25700000000000001}]},{"osd":2,"last update":"Fri Oct 6 00:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.22800000000000001,"5min":0.23899999999999999,"15min":0.23400000000000001},"min":{"1min":0.185,"5min":0.16700000000000001,"15min":0.16300000000000001},"max":{"1min":0.34899999999999998,"5min":0.40699999999999997,"15min":0.439},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.28999999999999998,"5min":0.29999999999999999,"15min":0.29699999999999999},"min":{"1min":0.252,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.40999999999999998,"5min":0.47499999999999998,"15min":0.53600000000000003},"last":0.28000000000000003}]}]},{"osd":1,"up_from":8,"seq":34359740681,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:18:35 2023","interfaces":[{"interface":"back","average":{"1min":0.23200000000000001,"5min":0.22500000000000001,"15min":0.20999999999999999},"min":{"1min":0.14999999999999999,"5min":0.14199999999999999,"15min":0.13100000000000001},"max":{"1min":0.31900000000000001,"5min":0.497,"15min":0.497},"last":0.17899999999999999},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.26000000000000001,"15min":0.26800000000000002},"min":{"1min":0.16300000000000001,"5min":0.154,"15min":0.154},"max":{"1min":0.34899999999999998,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.24099999999999999}]},{"osd":2,"last update":"Fri Oct 6 00:18:35 2023","interfaces":[{"interface":"back","average":{"1min":0.23699999999999999,"5min":0.23799999999999999,"15min":0.23300000000000001},"min":{"1min":0.18099999999999999,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":0.33400000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.20100000000000001},{"interface":"front","average":{"1min":0.245,"5min":0.26100000000000001,"15min":0.26800000000000002},"min":{"1min":0.19400000000000001,"5min":0.187,"15min":0.187},"max":{"1min":0.33300000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.25700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:18:47.759 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:18:47.759 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:18:47.760 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:18:47.760 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:18:47.760 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:18:47.760 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:18:47.760 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:18:47.761 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:18:47.761 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:18:47.761 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:18:47.903 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:47.901+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:48.152 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:48.151+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:48.881 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:48.879+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:49.160 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:49.159+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:49.875 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:49.874+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:50.165 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:50.163+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:50.922 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:50.920+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:51.206 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:51.204+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:51.876 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:51.874+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:52.197 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:52.195+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:52.897 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:52.895+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:53.203 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:53.201+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:53.947 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:53.945+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:54.220 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:54.218+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:54.987 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:54.985+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:55.200 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:55.199+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:56.014 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:56.013+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:56.184 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:56.183+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:56.990 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:56.988+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:57.146 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:57.144+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:57.961 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:57.959+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:58.116 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:58.115+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:58.984 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:58.983+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:59.110 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:18:59.109+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:18:59.972 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:18:59.970+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:00.108 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:00.107+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:00.983 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:00.981+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:01.105 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:01.105+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:02.024 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:02.023+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:02.137 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:02.136+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:02.985 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:02.984+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:03.113 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:03.112+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:03.944 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:03.944+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:04.091 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:04.091+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:04.925 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:04.925+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:05.138 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:05.138+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:05.973 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:05.973+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:06.168 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:06.168+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:06.992 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:06.991+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:07.214 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:07.214+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:07.763 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:19:08.009 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:08.008+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:08.138 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:19:08.141 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:19:08.152 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":5936,"stamp":"2023-10-06T00:19:06.137617+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002044"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":1964,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:18:56.184059+0000","last_change":"2023-10-06T00:18:56.184059+0000","last_active":"2023-10-06T00:18:56.184059+0000","last_peered":"2023-10-06T00:18:56.184059+0000","last_clean":"2023-10-06T00:18:56.184059+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:18:56.184059+0000","last_undegraded":"2023-10-06T00:18:56.184059+0000","last_fullsized":"2023-10-06T00:18:56.184059+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1867,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:18:54.219681+0000","last_change":"2023-10-06T00:18:54.219681+0000","last_active":"2023-10-06T00:18:54.219681+0000","last_peered":"2023-10-06T00:18:54.219681+0000","last_clean":"2023-10-06T00:18:54.219681+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:18:54.219681+0000","last_undegraded":"2023-10-06T00:18:54.219681+0000","last_fullsized":"2023-10-06T00:18:54.219681+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8617,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:16:00.340028+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:16:00.340028+0000","last_peered":"2023-10-06T00:16:00.340028+0000","last_clean":"2023-10-06T00:16:00.340028+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:16:00.340028+0000","last_undegraded":"2023-10-06T00:16:00.340028+0000","last_fullsized":"2023-10-06T00:16:00.340028+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 144s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11116,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:18:47.167921+0000","last_change":"2023-10-06T00:18:47.167921+0000","last_active":"2023-10-06T00:18:47.167921+0000","last_peered":"2023-10-06T00:18:47.167921+0000","last_clean":"2023-10-06T00:18:47.167921+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:18:47.167921+0000","last_undegraded":"2023-10-06T00:18:47.167921+0000","last_fullsized":"2023-10-06T00:18:47.167921+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2383,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:18:51.205835+0000","last_change":"2023-10-06T00:18:51.205835+0000","last_active":"2023-10-06T00:18:51.205835+0000","last_peered":"2023-10-06T00:18:51.205835+0000","last_clean":"2023-10-06T00:18:51.205835+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:18:51.205835+0000","last_undegraded":"2023-10-06T00:18:51.205835+0000","last_fullsized":"2023-10-06T00:18:51.205835+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11767,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:19:00.107864+0000","last_change":"2023-10-06T00:19:00.107864+0000","last_active":"2023-10-06T00:19:00.107864+0000","last_peered":"2023-10-06T00:19:00.107864+0000","last_clean":"2023-10-06T00:19:00.107864+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:19:00.107864+0000","last_undegraded":"2023-10-06T00:19:00.107864+0000","last_fullsized":"2023-10-06T00:19:00.107864+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11613,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:16:00.896651+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-06T00:16:00.896651+0000","last_peered":"2023-10-06T00:16:00.896651+0000","last_clean":"2023-10-06T00:16:00.896651+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-06T00:16:00.896651+0000","last_undegraded":"2023-10-06T00:16:00.896651+0000","last_fullsized":"2023-10-06T00:16:00.896651+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1783,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:16:01.247598+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-06T00:16:01.247598+0000","last_peered":"2023-10-06T00:16:01.247598+0000","last_clean":"2023-10-06T00:16:01.247598+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-06T00:16:01.247598+0000","last_undegraded":"2023-10-06T00:16:01.247598+0000","last_fullsized":"2023-10-06T00:16:01.247598+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1788,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:18:50.164701+0000","last_change":"2023-10-06T00:18:50.164701+0000","last_active":"2023-10-06T00:18:50.164701+0000","last_peered":"2023-10-06T00:18:50.164701+0000","last_clean":"2023-10-06T00:18:50.164701+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:18:50.164701+0000","last_undegraded":"2023-10-06T00:18:50.164701+0000","last_fullsized":"2023-10-06T00:18:50.164701+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740689,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:18:05 2023","interfaces":[{"interface":"back","average":{"1min":0.23200000000000001,"5min":0.23000000000000001,"15min":0.222},"min":{"1min":0.161,"5min":0.155,"15min":0.151},"max":{"1min":0.379,"5min":1.153,"15min":1.153},"last":0.185},{"interface":"front","average":{"1min":0.24099999999999999,"5min":0.23799999999999999,"15min":0.24299999999999999},"min":{"1min":0.17299999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.40100000000000002,"5min":0.41899999999999998,"15min":0.48399999999999999},"last":0.24299999999999999}]},{"osd":1,"last update":"Fri Oct 6 00:18:05 2023","interfaces":[{"interface":"back","average":{"1min":0.26300000000000001,"5min":0.255,"15min":0.246},"min":{"1min":0.184,"5min":0.16800000000000001,"15min":0.16800000000000001},"max":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.48699999999999999},"last":0.215},{"interface":"front","average":{"1min":0.29099999999999998,"5min":0.28000000000000003,"15min":0.27400000000000002},"min":{"1min":0.24099999999999999,"5min":0.217,"15min":0.16300000000000001},"max":{"1min":0.433,"5min":0.44900000000000001,"15min":0.51200000000000001},"last":0.28399999999999997}]}]},{"osd":0,"up_from":8,"seq":34359740690,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.19800000000000001,"5min":0.20499999999999999,"15min":0.20100000000000001},"min":{"1min":0.14699999999999999,"5min":0.13600000000000001,"15min":0.128},"max":{"1min":0.28999999999999998,"5min":0.34999999999999998,"15min":0.41299999999999998},"last":0.14799999999999999},{"interface":"front","average":{"1min":0.26500000000000001,"5min":0.27300000000000002,"15min":0.26500000000000001},"min":{"1min":0.22,"5min":0.186,"15min":0.152},"max":{"1min":0.38,"5min":0.443,"15min":0.47899999999999998},"last":0.21299999999999999}]},{"osd":2,"last update":"Fri Oct 6 00:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.22800000000000001,"5min":0.23899999999999999,"15min":0.23400000000000001},"min":{"1min":0.185,"5min":0.16700000000000001,"15min":0.16300000000000001},"max":{"1min":0.34899999999999998,"5min":0.40699999999999997,"15min":0.439},"last":0.182},{"interface":"front","average":{"1min":0.28999999999999998,"5min":0.29999999999999999,"15min":0.29699999999999999},"min":{"1min":0.252,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.40999999999999998,"5min":0.47499999999999998,"15min":0.53600000000000003},"last":0.249}]}]},{"osd":1,"up_from":8,"seq":34359740685,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:18:35 2023","interfaces":[{"interface":"back","average":{"1min":0.23200000000000001,"5min":0.22500000000000001,"15min":0.20999999999999999},"min":{"1min":0.14999999999999999,"5min":0.14199999999999999,"15min":0.13100000000000001},"max":{"1min":0.31900000000000001,"5min":0.497,"15min":0.497},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.26000000000000001,"15min":0.26800000000000002},"min":{"1min":0.16300000000000001,"5min":0.154,"15min":0.154},"max":{"1min":0.34899999999999998,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.41199999999999998}]},{"osd":2,"last update":"Fri Oct 6 00:18:35 2023","interfaces":[{"interface":"back","average":{"1min":0.23699999999999999,"5min":0.23799999999999999,"15min":0.23300000000000001},"min":{"1min":0.18099999999999999,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":0.33400000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.245,"5min":0.26100000000000001,"15min":0.26800000000000002},"min":{"1min":0.19400000000000001,"5min":0.187,"15min":0.187},"max":{"1min":0.33300000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.371}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:19:08.154 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:19:08.154 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:19:08.154 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:19:08.155 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:19:08.155 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:19:08.155 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:19:08.155 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:19:08.155 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:19:08.155 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:19:08.156 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:19:08.235 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:08.234+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:09.008 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:09.008+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:09.260 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:09.260+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:10.057 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:10.057+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:10.274 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:10.274+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:11.041 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:11.040+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:11.318 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:11.317+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:12.065 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:12.064+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:12.337 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:12.336+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:13.057 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:13.056+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:13.367 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:13.366+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:14.073 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:14.072+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:14.406 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:14.406+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:15.065 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:15.064+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:15.363 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:15.362+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:16.104 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:16.104+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:16.351 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:16.350+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:17.072 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:17.071+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:17.358 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:17.357+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:18.105 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:18.104+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:18.352 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:18.352+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:19.131 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:19.130+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:19.314 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:19.313+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:20.113 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:20.112+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:20.298 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:20.297+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:21.110 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:21.109+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:21.297 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:21.296+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:22.074 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:22.072+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:22.290 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:22.288+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:23.081 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:23.079+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:23.329 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:23.328+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:24.063 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:24.061+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:24.312 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:24.311+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:25.077 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:25.075+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:25.282 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:25.281+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:26.114 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:26.112+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:26.251 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:26.250+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:27.159 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:27.158+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:27.270 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:27.268+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:28.132 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:28.131+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:28.157 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:19:28.317 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:28.315+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:28.535 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:19:28.537 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:19:28.548 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":5947,"stamp":"2023-10-06T00:19:28.140643+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001667"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":1967,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:19:20.297790+0000","last_change":"2023-10-06T00:19:20.297790+0000","last_active":"2023-10-06T00:19:20.297790+0000","last_peered":"2023-10-06T00:19:20.297790+0000","last_clean":"2023-10-06T00:19:20.297790+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:19:20.297790+0000","last_undegraded":"2023-10-06T00:19:20.297790+0000","last_fullsized":"2023-10-06T00:19:20.297790+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1870,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:19:17.357487+0000","last_change":"2023-10-06T00:19:17.357487+0000","last_active":"2023-10-06T00:19:17.357487+0000","last_peered":"2023-10-06T00:19:17.357487+0000","last_clean":"2023-10-06T00:19:17.357487+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:19:17.357487+0000","last_undegraded":"2023-10-06T00:19:17.357487+0000","last_fullsized":"2023-10-06T00:19:17.357487+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8617,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:16:00.340028+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:16:00.340028+0000","last_peered":"2023-10-06T00:16:00.340028+0000","last_clean":"2023-10-06T00:16:00.340028+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:16:00.340028+0000","last_undegraded":"2023-10-06T00:16:00.340028+0000","last_fullsized":"2023-10-06T00:16:00.340028+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 144s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11122,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:19:22.289416+0000","last_change":"2023-10-06T00:19:22.289416+0000","last_active":"2023-10-06T00:19:22.289416+0000","last_peered":"2023-10-06T00:19:22.289416+0000","last_clean":"2023-10-06T00:19:22.289416+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:19:22.289416+0000","last_undegraded":"2023-10-06T00:19:22.289416+0000","last_fullsized":"2023-10-06T00:19:22.289416+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2386,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:19:14.406036+0000","last_change":"2023-10-06T00:19:14.406036+0000","last_active":"2023-10-06T00:19:14.406036+0000","last_peered":"2023-10-06T00:19:14.406036+0000","last_clean":"2023-10-06T00:19:14.406036+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:19:14.406036+0000","last_undegraded":"2023-10-06T00:19:14.406036+0000","last_fullsized":"2023-10-06T00:19:14.406036+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11770,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:19:21.296580+0000","last_change":"2023-10-06T00:19:21.296580+0000","last_active":"2023-10-06T00:19:21.296580+0000","last_peered":"2023-10-06T00:19:21.296580+0000","last_clean":"2023-10-06T00:19:21.296580+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:19:21.296580+0000","last_undegraded":"2023-10-06T00:19:21.296580+0000","last_fullsized":"2023-10-06T00:19:21.296580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11613,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:16:00.896651+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-06T00:16:00.896651+0000","last_peered":"2023-10-06T00:16:00.896651+0000","last_clean":"2023-10-06T00:16:00.896651+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-06T00:16:00.896651+0000","last_undegraded":"2023-10-06T00:16:00.896651+0000","last_fullsized":"2023-10-06T00:16:00.896651+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1783,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:16:01.247598+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-06T00:16:01.247598+0000","last_peered":"2023-10-06T00:16:01.247598+0000","last_clean":"2023-10-06T00:16:01.247598+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-06T00:16:01.247598+0000","last_undegraded":"2023-10-06T00:16:01.247598+0000","last_fullsized":"2023-10-06T00:16:01.247598+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1791,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:19:13.367039+0000","last_change":"2023-10-06T00:19:13.367039+0000","last_active":"2023-10-06T00:19:13.367039+0000","last_peered":"2023-10-06T00:19:13.367039+0000","last_clean":"2023-10-06T00:19:13.367039+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:19:13.367039+0000","last_undegraded":"2023-10-06T00:19:13.367039+0000","last_fullsized":"2023-10-06T00:19:13.367039+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740693,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:19:09 2023","interfaces":[{"interface":"back","average":{"1min":0.20399999999999999,"5min":0.22800000000000001,"15min":0.221},"min":{"1min":0.14699999999999999,"5min":0.14699999999999999,"15min":0.14699999999999999},"max":{"1min":0.38800000000000001,"5min":1.153,"15min":1.153},"last":0.19900000000000001},{"interface":"front","average":{"1min":0.224,"5min":0.23499999999999999,"15min":0.24099999999999999},"min":{"1min":0.17299999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.34200000000000003,"5min":0.41899999999999998,"15min":0.48399999999999999},"last":0.24199999999999999}]},{"osd":1,"last update":"Fri Oct 6 00:19:09 2023","interfaces":[{"interface":"back","average":{"1min":0.24099999999999999,"5min":0.254,"15min":0.247},"min":{"1min":0.157,"5min":0.157,"15min":0.157},"max":{"1min":0.372,"5min":0.44500000000000001,"15min":0.48699999999999999},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.27600000000000002,"5min":0.28000000000000003,"15min":0.27400000000000002},"min":{"1min":0.218,"5min":0.218,"15min":0.16300000000000001},"max":{"1min":0.434,"5min":0.44900000000000001,"15min":0.51200000000000001},"last":0.30199999999999999}]}]},{"osd":0,"up_from":8,"seq":34359740694,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.187,"5min":0.19900000000000001,"15min":0.19900000000000001},"min":{"1min":0.14799999999999999,"5min":0.13600000000000001,"15min":0.128},"max":{"1min":0.23899999999999999,"5min":0.34999999999999998,"15min":0.41299999999999998},"last":0.16600000000000001},{"interface":"front","average":{"1min":0.251,"5min":0.26600000000000001,"15min":0.26500000000000001},"min":{"1min":0.21299999999999999,"5min":0.186,"15min":0.152},"max":{"1min":0.30299999999999999,"5min":0.443,"15min":0.47899999999999998},"last":0.25}]},{"osd":2,"last update":"Fri Oct 6 00:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.23300000000000001,"5min":0.23499999999999999,"15min":0.23400000000000001},"min":{"1min":0.182,"5min":0.16700000000000001,"15min":0.16300000000000001},"max":{"1min":0.28100000000000003,"5min":0.40699999999999997,"15min":0.439},"last":0.21299999999999999},{"interface":"front","average":{"1min":0.28599999999999998,"5min":0.29399999999999998,"15min":0.29699999999999999},"min":{"1min":0.249,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.33500000000000002,"5min":0.47499999999999998,"15min":0.53600000000000003},"last":0.28299999999999997}]}]},{"osd":1,"up_from":8,"seq":34359740689,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:18:35 2023","interfaces":[{"interface":"back","average":{"1min":0.23200000000000001,"5min":0.22500000000000001,"15min":0.20999999999999999},"min":{"1min":0.14999999999999999,"5min":0.14199999999999999,"15min":0.13100000000000001},"max":{"1min":0.31900000000000001,"5min":0.497,"15min":0.497},"last":0.17399999999999999},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.26000000000000001,"15min":0.26800000000000002},"min":{"1min":0.16300000000000001,"5min":0.154,"15min":0.154},"max":{"1min":0.34899999999999998,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.254}]},{"osd":2,"last update":"Fri Oct 6 00:18:35 2023","interfaces":[{"interface":"back","average":{"1min":0.23699999999999999,"5min":0.23799999999999999,"15min":0.23300000000000001},"min":{"1min":0.18099999999999999,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":0.33400000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"last":0.219},{"interface":"front","average":{"1min":0.245,"5min":0.26100000000000001,"15min":0.26800000000000002},"min":{"1min":0.19400000000000001,"5min":0.187,"15min":0.187},"max":{"1min":0.33300000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.26700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:19:28.549 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:19:28.549 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:19:28.549 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:19:28.550 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:19:28.550 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:19:28.550 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:19:28.550 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:19:28.550 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:19:28.551 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:19:28.551 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:19:29.089 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:29.087+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:29.280 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:29.279+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:30.133 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:30.132+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:30.259 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:30.257+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:31.143 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:31.141+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:31.265 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:31.263+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:32.159 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:32.158+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:32.275 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:32.274+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:33.142 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:33.140+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:33.248 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:33.246+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:34.163 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:34.161+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:34.290 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:34.289+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:35.162 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:35.161+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:35.241 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:35.240+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:36.192 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:36.190+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:36.282 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:36.280+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:37.148 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:37.147+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:37.244 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:37.243+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:38.174 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:38.172+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:38.214 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:38.213+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:39.131 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:39.129+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:39.199 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:39.197+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:40.111 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:40.110+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:40.161 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:40.160+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:41.107 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:41.105+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:41.129 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:41.127+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:42.078 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:42.076+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:42.085 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:42.083+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:43.049 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:43.047+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:43.096 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:43.095+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:44.027 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:44.025+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:44.101 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:44.100+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:45.041 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:45.039+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:45.060 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:45.058+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:46.074 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:46.072+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:46.084 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:46.083+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:47.031 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:47.029+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:47.088 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:47.086+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:48.009 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:48.007+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:48.065 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:48.063+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:48.551 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:19:48.993 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:19:48.996 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:19:49.007 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":5957,"stamp":"2023-10-06T00:19:48.143613+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001830"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":1970,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:19:34.290246+0000","last_change":"2023-10-06T00:19:34.290246+0000","last_active":"2023-10-06T00:19:34.290246+0000","last_peered":"2023-10-06T00:19:34.290246+0000","last_clean":"2023-10-06T00:19:34.290246+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:19:34.290246+0000","last_undegraded":"2023-10-06T00:19:34.290246+0000","last_fullsized":"2023-10-06T00:19:34.290246+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:19:31.264368+0000","last_change":"2023-10-06T00:19:31.264368+0000","last_active":"2023-10-06T00:19:31.264368+0000","last_peered":"2023-10-06T00:19:31.264368+0000","last_clean":"2023-10-06T00:19:31.264368+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:19:31.264368+0000","last_undegraded":"2023-10-06T00:19:31.264368+0000","last_fullsized":"2023-10-06T00:19:31.264368+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8617,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:16:00.340028+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:16:00.340028+0000","last_peered":"2023-10-06T00:16:00.340028+0000","last_clean":"2023-10-06T00:16:00.340028+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:16:00.340028+0000","last_undegraded":"2023-10-06T00:16:00.340028+0000","last_fullsized":"2023-10-06T00:16:00.340028+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 144s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11122,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:19:22.289416+0000","last_change":"2023-10-06T00:19:22.289416+0000","last_active":"2023-10-06T00:19:22.289416+0000","last_peered":"2023-10-06T00:19:22.289416+0000","last_clean":"2023-10-06T00:19:22.289416+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:19:22.289416+0000","last_undegraded":"2023-10-06T00:19:22.289416+0000","last_fullsized":"2023-10-06T00:19:22.289416+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2389,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:19:27.269792+0000","last_change":"2023-10-06T00:19:27.269792+0000","last_active":"2023-10-06T00:19:27.269792+0000","last_peered":"2023-10-06T00:19:27.269792+0000","last_clean":"2023-10-06T00:19:27.269792+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:19:27.269792+0000","last_undegraded":"2023-10-06T00:19:27.269792+0000","last_fullsized":"2023-10-06T00:19:27.269792+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11773,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:19:42.084427+0000","last_change":"2023-10-06T00:19:42.084427+0000","last_active":"2023-10-06T00:19:42.084427+0000","last_peered":"2023-10-06T00:19:42.084427+0000","last_clean":"2023-10-06T00:19:42.084427+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:19:42.084427+0000","last_undegraded":"2023-10-06T00:19:42.084427+0000","last_fullsized":"2023-10-06T00:19:42.084427+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11613,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:16:00.896651+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-06T00:16:00.896651+0000","last_peered":"2023-10-06T00:16:00.896651+0000","last_clean":"2023-10-06T00:16:00.896651+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-06T00:16:00.896651+0000","last_undegraded":"2023-10-06T00:16:00.896651+0000","last_fullsized":"2023-10-06T00:16:00.896651+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1783,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:16:01.247598+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-06T00:16:01.247598+0000","last_peered":"2023-10-06T00:16:01.247598+0000","last_clean":"2023-10-06T00:16:01.247598+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-06T00:16:01.247598+0000","last_undegraded":"2023-10-06T00:16:01.247598+0000","last_fullsized":"2023-10-06T00:16:01.247598+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1794,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:19:24.312475+0000","last_change":"2023-10-06T00:19:24.312475+0000","last_active":"2023-10-06T00:19:24.312475+0000","last_peered":"2023-10-06T00:19:24.312475+0000","last_clean":"2023-10-06T00:19:24.312475+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:19:24.312475+0000","last_undegraded":"2023-10-06T00:19:24.312475+0000","last_fullsized":"2023-10-06T00:19:24.312475+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740697,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:19:09 2023","interfaces":[{"interface":"back","average":{"1min":0.20399999999999999,"5min":0.22800000000000001,"15min":0.221},"min":{"1min":0.14699999999999999,"5min":0.14699999999999999,"15min":0.14699999999999999},"max":{"1min":0.38800000000000001,"5min":1.153,"15min":1.153},"last":0.18099999999999999},{"interface":"front","average":{"1min":0.224,"5min":0.23499999999999999,"15min":0.24099999999999999},"min":{"1min":0.17299999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.34200000000000003,"5min":0.41899999999999998,"15min":0.48399999999999999},"last":1.1659999999999999}]},{"osd":1,"last update":"Fri Oct 6 00:19:09 2023","interfaces":[{"interface":"back","average":{"1min":0.24099999999999999,"5min":0.254,"15min":0.247},"min":{"1min":0.157,"5min":0.157,"15min":0.157},"max":{"1min":0.372,"5min":0.44500000000000001,"15min":0.48699999999999999},"last":0.22600000000000001},{"interface":"front","average":{"1min":0.27600000000000002,"5min":0.28000000000000003,"15min":0.27400000000000002},"min":{"1min":0.218,"5min":0.218,"15min":0.16300000000000001},"max":{"1min":0.434,"5min":0.44900000000000001,"15min":0.51200000000000001},"last":1.2110000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740698,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.187,"5min":0.19900000000000001,"15min":0.19900000000000001},"min":{"1min":0.14799999999999999,"5min":0.13600000000000001,"15min":0.128},"max":{"1min":0.23899999999999999,"5min":0.34999999999999998,"15min":0.41299999999999998},"last":0.182},{"interface":"front","average":{"1min":0.251,"5min":0.26600000000000001,"15min":0.26500000000000001},"min":{"1min":0.21299999999999999,"5min":0.186,"15min":0.152},"max":{"1min":0.30299999999999999,"5min":0.443,"15min":0.47899999999999998},"last":0.26500000000000001}]},{"osd":2,"last update":"Fri Oct 6 00:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.23300000000000001,"5min":0.23499999999999999,"15min":0.23400000000000001},"min":{"1min":0.182,"5min":0.16700000000000001,"15min":0.16300000000000001},"max":{"1min":0.28100000000000003,"5min":0.40699999999999997,"15min":0.439},"last":0.223},{"interface":"front","average":{"1min":0.28599999999999998,"5min":0.29399999999999998,"15min":0.29699999999999999},"min":{"1min":0.249,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.33500000000000002,"5min":0.47499999999999998,"15min":0.53600000000000003},"last":0.30299999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740693,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.20899999999999999,"5min":0.222,"15min":0.21099999999999999},"min":{"1min":0.13100000000000001,"5min":0.13100000000000001,"15min":0.13100000000000001},"max":{"1min":0.35499999999999998,"5min":0.41399999999999998,"15min":0.497},"last":0.20300000000000001},{"interface":"front","average":{"1min":0.251,"5min":0.25800000000000001,"15min":0.26600000000000001},"min":{"1min":0.17399999999999999,"5min":0.16300000000000001,"15min":0.154},"max":{"1min":0.41199999999999998,"5min":0.47599999999999998,"15min":0.51100000000000001},"last":0.17199999999999999}]},{"osd":2,"last update":"Fri Oct 6 00:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.23699999999999999,"5min":0.23799999999999999,"15min":0.23400000000000001},"min":{"1min":0.189,"5min":0.17599999999999999,"15min":0.17100000000000001},"max":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.46999999999999997},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.24399999999999999,"5min":0.254,"15min":0.26600000000000001},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.371,"5min":0.50900000000000001,"15min":0.54700000000000004},"last":0.217}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:19:49.009 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:19:49.009 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:19:49.009 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:19:49.010 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:19:49.010 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:19:49.010 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:19:49.010 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:19:49.010 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:19:49.011 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:19:49.011 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:19:49.048 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:49.046+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:49.048 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:49.046+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:50.009 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:50.007+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:50.065 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:50.061+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:50.984 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:50.983+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:51.061 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:51.059+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:51.966 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:51.964+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:52.095 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:52.094+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:53.001 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:52.999+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:53.143 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:53.142+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:54.001 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:54.000+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:54.185 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:54.183+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:54.992 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:54.991+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:55.235 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:55.233+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:55.960 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:55.959+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:56.222 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:56.221+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:56.929 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:56.927+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:57.207 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:57.205+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:57.924 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:57.922+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:58.186 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:58.184+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:58.914 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:58.914+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:59.178 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:19:59.177+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:19:59.957 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:19:59.957+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:00.141 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:00.140+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:00.950 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:00.950+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:01.145 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:01.145+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:01.957 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:01.957+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:02.113 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:02.112+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:02.990 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:02.989+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:03.142 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:03.141+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:03.977 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:03.977+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:04.166 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:04.166+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:04.979 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:04.979+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:05.121 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:05.120+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:05.981 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:05.980+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:06.089 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:06.089+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:07.011 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:07.011+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:07.119 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:07.119+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:08.060 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:08.060+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:08.155 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:08.155+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:09.012 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:20:09.015 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:09.015+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:09.172 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:09.171+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:09.364 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:20:09.366 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:20:09.378 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":5967,"stamp":"2023-10-06T00:20:08.146506+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001747"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":1973,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:19:58.185779+0000","last_change":"2023-10-06T00:19:58.185779+0000","last_active":"2023-10-06T00:19:58.185779+0000","last_peered":"2023-10-06T00:19:58.185779+0000","last_clean":"2023-10-06T00:19:58.185779+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:19:58.185779+0000","last_undegraded":"2023-10-06T00:19:58.185779+0000","last_fullsized":"2023-10-06T00:19:58.185779+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1876,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:19:55.234325+0000","last_change":"2023-10-06T00:19:55.234325+0000","last_active":"2023-10-06T00:19:55.234325+0000","last_peered":"2023-10-06T00:19:55.234325+0000","last_clean":"2023-10-06T00:19:55.234325+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:19:55.234325+0000","last_undegraded":"2023-10-06T00:19:55.234325+0000","last_fullsized":"2023-10-06T00:19:55.234325+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8617,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:16:00.340028+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:16:00.340028+0000","last_peered":"2023-10-06T00:16:00.340028+0000","last_clean":"2023-10-06T00:16:00.340028+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:16:00.340028+0000","last_undegraded":"2023-10-06T00:16:00.340028+0000","last_fullsized":"2023-10-06T00:16:00.340028+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 144s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11128,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:20:03.141437+0000","last_change":"2023-10-06T00:20:03.141437+0000","last_active":"2023-10-06T00:20:03.141437+0000","last_peered":"2023-10-06T00:20:03.141437+0000","last_clean":"2023-10-06T00:20:03.141437+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:20:03.141437+0000","last_undegraded":"2023-10-06T00:20:03.141437+0000","last_fullsized":"2023-10-06T00:20:03.141437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2392,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:19:54.184483+0000","last_change":"2023-10-06T00:19:54.184483+0000","last_active":"2023-10-06T00:19:54.184483+0000","last_peered":"2023-10-06T00:19:54.184483+0000","last_clean":"2023-10-06T00:19:54.184483+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:19:54.184483+0000","last_undegraded":"2023-10-06T00:19:54.184483+0000","last_fullsized":"2023-10-06T00:19:54.184483+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11776,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:19:59.178223+0000","last_change":"2023-10-06T00:19:59.178223+0000","last_active":"2023-10-06T00:19:59.178223+0000","last_peered":"2023-10-06T00:19:59.178223+0000","last_clean":"2023-10-06T00:19:59.178223+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:19:59.178223+0000","last_undegraded":"2023-10-06T00:19:59.178223+0000","last_fullsized":"2023-10-06T00:19:59.178223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11613,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:16:00.896651+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-06T00:16:00.896651+0000","last_peered":"2023-10-06T00:16:00.896651+0000","last_clean":"2023-10-06T00:16:00.896651+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-06T00:16:00.896651+0000","last_undegraded":"2023-10-06T00:16:00.896651+0000","last_fullsized":"2023-10-06T00:16:00.896651+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1783,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:16:01.247598+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-06T00:16:01.247598+0000","last_peered":"2023-10-06T00:16:01.247598+0000","last_clean":"2023-10-06T00:16:01.247598+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-06T00:16:01.247598+0000","last_undegraded":"2023-10-06T00:16:01.247598+0000","last_fullsized":"2023-10-06T00:16:01.247598+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1797,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:19:53.142913+0000","last_change":"2023-10-06T00:19:53.142913+0000","last_active":"2023-10-06T00:19:53.142913+0000","last_peered":"2023-10-06T00:19:53.142913+0000","last_clean":"2023-10-06T00:19:53.142913+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:19:53.142913+0000","last_undegraded":"2023-10-06T00:19:53.142913+0000","last_fullsized":"2023-10-06T00:19:53.142913+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740701,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:19:09 2023","interfaces":[{"interface":"back","average":{"1min":0.20399999999999999,"5min":0.22800000000000001,"15min":0.221},"min":{"1min":0.14699999999999999,"5min":0.14699999999999999,"15min":0.14699999999999999},"max":{"1min":0.38800000000000001,"5min":1.153,"15min":1.153},"last":0.18099999999999999},{"interface":"front","average":{"1min":0.224,"5min":0.23499999999999999,"15min":0.24099999999999999},"min":{"1min":0.17299999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.34200000000000003,"5min":0.41899999999999998,"15min":0.48399999999999999},"last":0.216}]},{"osd":1,"last update":"Fri Oct 6 00:19:09 2023","interfaces":[{"interface":"back","average":{"1min":0.24099999999999999,"5min":0.254,"15min":0.247},"min":{"1min":0.157,"5min":0.157,"15min":0.157},"max":{"1min":0.372,"5min":0.44500000000000001,"15min":0.48699999999999999},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.27600000000000002,"5min":0.28000000000000003,"15min":0.27400000000000002},"min":{"1min":0.218,"5min":0.218,"15min":0.16300000000000001},"max":{"1min":0.434,"5min":0.44900000000000001,"15min":0.51200000000000001},"last":0.27900000000000003}]}]},{"osd":0,"up_from":8,"seq":34359740702,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.187,"5min":0.19900000000000001,"15min":0.19900000000000001},"min":{"1min":0.14799999999999999,"5min":0.13600000000000001,"15min":0.128},"max":{"1min":0.23899999999999999,"5min":0.34999999999999998,"15min":0.41299999999999998},"last":0.17000000000000001},{"interface":"front","average":{"1min":0.251,"5min":0.26600000000000001,"15min":0.26500000000000001},"min":{"1min":0.21299999999999999,"5min":0.186,"15min":0.152},"max":{"1min":0.30299999999999999,"5min":0.443,"15min":0.47899999999999998},"last":0.27300000000000002}]},{"osd":2,"last update":"Fri Oct 6 00:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.23300000000000001,"5min":0.23499999999999999,"15min":0.23400000000000001},"min":{"1min":0.182,"5min":0.16700000000000001,"15min":0.16300000000000001},"max":{"1min":0.28100000000000003,"5min":0.40699999999999997,"15min":0.439},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.28599999999999998,"5min":0.29399999999999998,"15min":0.29699999999999999},"min":{"1min":0.249,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.33500000000000002,"5min":0.47499999999999998,"15min":0.53600000000000003},"last":0.30099999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740697,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.20899999999999999,"5min":0.222,"15min":0.21099999999999999},"min":{"1min":0.13100000000000001,"5min":0.13100000000000001,"15min":0.13100000000000001},"max":{"1min":0.35499999999999998,"5min":0.41399999999999998,"15min":0.497},"last":0.371},{"interface":"front","average":{"1min":0.251,"5min":0.25800000000000001,"15min":0.26600000000000001},"min":{"1min":0.17399999999999999,"5min":0.16300000000000001,"15min":0.154},"max":{"1min":0.41199999999999998,"5min":0.47599999999999998,"15min":0.51100000000000001},"last":0.315}]},{"osd":2,"last update":"Fri Oct 6 00:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.23699999999999999,"5min":0.23799999999999999,"15min":0.23400000000000001},"min":{"1min":0.189,"5min":0.17599999999999999,"15min":0.17100000000000001},"max":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.46999999999999997},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.24399999999999999,"5min":0.254,"15min":0.26600000000000001},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.371,"5min":0.50900000000000001,"15min":0.54700000000000004},"last":0.35399999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:20:09.380 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:20:09.380 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:20:09.380 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:20:09.380 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:20:09.381 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:20:09.381 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:20:09.381 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:20:09.381 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:20:09.381 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:20:09.382 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:20:10.008 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:10.006+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:10.153 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:10.153+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:10.977 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:10.977+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:11.160 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:11.159+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:11.935 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:11.934+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:12.143 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:12.142+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:12.926 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:12.925+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:13.181 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:13.180+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:13.941 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:13.940+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:14.137 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:14.136+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:14.946 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:14.945+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:15.136 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:15.136+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:15.906 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:15.906+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:16.152 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:16.152+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:16.881 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:16.881+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:17.170 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:17.170+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:17.871 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:17.871+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:18.177 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:18.176+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:18.919 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:18.919+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:19.188 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:19.187+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:19.900 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:19.898+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:20.176 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:20.174+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:20.897 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:20.896+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:21.179 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:21.178+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:21.869 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:21.868+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:22.156 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:22.154+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:22.892 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:22.890+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:23.172 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:23.170+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:23.925 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:23.923+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:24.151 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:24.149+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:24.923 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:24.921+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:25.104 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:25.102+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:25.930 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:25.929+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:26.150 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:26.148+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:26.940 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:26.938+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:27.124 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:27.122+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:27.932 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:27.930+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:28.115 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:28.114+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:28.971 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:28.970+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:29.120 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:29.118+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:29.382 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:20:29.727 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:20:29.729 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:20:29.741 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":5977,"stamp":"2023-10-06T00:20:28.149259+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001643"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":1973,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:19:58.185779+0000","last_change":"2023-10-06T00:19:58.185779+0000","last_active":"2023-10-06T00:19:58.185779+0000","last_peered":"2023-10-06T00:19:58.185779+0000","last_clean":"2023-10-06T00:19:58.185779+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:19:58.185779+0000","last_undegraded":"2023-10-06T00:19:58.185779+0000","last_fullsized":"2023-10-06T00:19:58.185779+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1879,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:20:16.152391+0000","last_change":"2023-10-06T00:20:16.152391+0000","last_active":"2023-10-06T00:20:16.152391+0000","last_peered":"2023-10-06T00:20:16.152391+0000","last_clean":"2023-10-06T00:20:16.152391+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:20:16.152391+0000","last_undegraded":"2023-10-06T00:20:16.152391+0000","last_fullsized":"2023-10-06T00:20:16.152391+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8617,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:16:00.340028+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:16:00.340028+0000","last_peered":"2023-10-06T00:16:00.340028+0000","last_clean":"2023-10-06T00:16:00.340028+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:16:00.340028+0000","last_undegraded":"2023-10-06T00:16:00.340028+0000","last_fullsized":"2023-10-06T00:16:00.340028+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 144s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11128,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:20:03.141437+0000","last_change":"2023-10-06T00:20:03.141437+0000","last_active":"2023-10-06T00:20:03.141437+0000","last_peered":"2023-10-06T00:20:03.141437+0000","last_clean":"2023-10-06T00:20:03.141437+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:20:03.141437+0000","last_undegraded":"2023-10-06T00:20:03.141437+0000","last_fullsized":"2023-10-06T00:20:03.141437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2395,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:20:07.119131+0000","last_change":"2023-10-06T00:20:07.119131+0000","last_active":"2023-10-06T00:20:07.119131+0000","last_peered":"2023-10-06T00:20:07.119131+0000","last_clean":"2023-10-06T00:20:07.119131+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:20:07.119131+0000","last_undegraded":"2023-10-06T00:20:07.119131+0000","last_fullsized":"2023-10-06T00:20:07.119131+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11776,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:19:59.178223+0000","last_change":"2023-10-06T00:19:59.178223+0000","last_active":"2023-10-06T00:19:59.178223+0000","last_peered":"2023-10-06T00:19:59.178223+0000","last_clean":"2023-10-06T00:19:59.178223+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:19:59.178223+0000","last_undegraded":"2023-10-06T00:19:59.178223+0000","last_fullsized":"2023-10-06T00:19:59.178223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11613,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:16:00.896651+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-06T00:16:00.896651+0000","last_peered":"2023-10-06T00:16:00.896651+0000","last_clean":"2023-10-06T00:16:00.896651+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-06T00:16:00.896651+0000","last_undegraded":"2023-10-06T00:16:00.896651+0000","last_fullsized":"2023-10-06T00:16:00.896651+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1783,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:16:01.247598+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-06T00:16:01.247598+0000","last_peered":"2023-10-06T00:16:01.247598+0000","last_clean":"2023-10-06T00:16:01.247598+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-06T00:16:01.247598+0000","last_undegraded":"2023-10-06T00:16:01.247598+0000","last_fullsized":"2023-10-06T00:16:01.247598+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1800,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:20:04.166097+0000","last_change":"2023-10-06T00:20:04.166097+0000","last_active":"2023-10-06T00:20:04.166097+0000","last_peered":"2023-10-06T00:20:04.166097+0000","last_clean":"2023-10-06T00:20:04.166097+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:20:04.166097+0000","last_undegraded":"2023-10-06T00:20:04.166097+0000","last_fullsized":"2023-10-06T00:20:04.166097+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740706,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.20100000000000001,"5min":0.22500000000000001,"15min":0.22},"min":{"1min":0.13800000000000001,"5min":0.13800000000000001,"15min":0.13800000000000001},"max":{"1min":0.52100000000000002,"5min":1.153,"15min":1.153},"last":0.182},{"interface":"front","average":{"1min":0.27300000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"min":{"1min":0.17999999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.214}]},{"osd":1,"last update":"Fri Oct 6 00:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.253,"5min":0.255,"15min":0.248},"min":{"1min":0.187,"5min":0.157,"15min":0.157},"max":{"1min":0.35299999999999998,"5min":0.44500000000000001,"15min":0.48699999999999999},"last":0.20499999999999999},{"interface":"front","average":{"1min":0.312,"5min":0.28999999999999998,"15min":0.27700000000000002},"min":{"1min":0.223,"5min":0.218,"15min":0.19600000000000001},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.25600000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740706,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.193,"5min":0.19900000000000001,"15min":0.19800000000000001},"min":{"1min":0.14899999999999999,"5min":0.14399999999999999,"15min":0.128},"max":{"1min":0.29499999999999998,"5min":0.34999999999999998,"15min":0.41299999999999998},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.26500000000000001,"15min":0.26400000000000001},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.159},"max":{"1min":0.312,"5min":0.443,"15min":0.443},"last":0.34200000000000003}]},{"osd":2,"last update":"Fri Oct 6 00:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.22800000000000001,"5min":0.23499999999999999,"15min":0.23300000000000001},"min":{"1min":0.20000000000000001,"5min":0.17100000000000001,"15min":0.16300000000000001},"max":{"1min":0.27800000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"min":{"1min":0.24199999999999999,"5min":0.215,"15min":0.21299999999999999},"max":{"1min":0.34999999999999998,"5min":0.47499999999999998,"15min":0.53600000000000003},"last":0.375}]}]},{"osd":1,"up_from":8,"seq":34359740701,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.20899999999999999,"5min":0.222,"15min":0.21099999999999999},"min":{"1min":0.13100000000000001,"5min":0.13100000000000001,"15min":0.13100000000000001},"max":{"1min":0.35499999999999998,"5min":0.41399999999999998,"15min":0.497},"last":0.191},{"interface":"front","average":{"1min":0.251,"5min":0.25800000000000001,"15min":0.26600000000000001},"min":{"1min":0.17399999999999999,"5min":0.16300000000000001,"15min":0.154},"max":{"1min":0.41199999999999998,"5min":0.47599999999999998,"15min":0.51100000000000001},"last":0.26400000000000001}]},{"osd":2,"last update":"Fri Oct 6 00:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.23699999999999999,"5min":0.23799999999999999,"15min":0.23400000000000001},"min":{"1min":0.189,"5min":0.17599999999999999,"15min":0.17100000000000001},"max":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.46999999999999997},"last":0.247},{"interface":"front","average":{"1min":0.24399999999999999,"5min":0.254,"15min":0.26600000000000001},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.371,"5min":0.50900000000000001,"15min":0.54700000000000004},"last":0.23400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:20:29.743 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:20:29.743 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:20:29.743 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:20:29.744 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:20:29.744 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:20:29.744 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:20:29.744 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:20:29.744 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:20:29.745 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:20:29.745 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.7 2023-10-06T00:20:30.010 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:30.008+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:30.079 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 2.7 on osd.1 to deep-scrub 2023-10-06T00:20:30.089 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.6 2023-10-06T00:20:30.154 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:30.153+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:30.440 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 2.6 on osd.1 to deep-scrub 2023-10-06T00:20:30.451 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.1 2023-10-06T00:20:30.816 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 3.1 on osd.0 to deep-scrub 2023-10-06T00:20:30.827 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.2 2023-10-06T00:20:31.013 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:31.012+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:31.162 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 3.2 on osd.1 to deep-scrub 2023-10-06T00:20:31.173 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.3 2023-10-06T00:20:31.195 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:31.194+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:31.500 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 2.3 on osd.1 to deep-scrub 2023-10-06T00:20:31.512 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.0 2023-10-06T00:20:31.880 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 3.0 on osd.1 to deep-scrub 2023-10-06T00:20:31.892 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.3 2023-10-06T00:20:32.053 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:32.051+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:32.213 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:32.212+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:32.248 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 3.3 on osd.0 to deep-scrub 2023-10-06T00:20:32.260 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.2 2023-10-06T00:20:32.592 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 2.2 on osd.0 to deep-scrub 2023-10-06T00:20:32.603 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.4 2023-10-06T00:20:32.949 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 2.4 on osd.1 to deep-scrub 2023-10-06T00:20:32.959 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:20:33.028 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:33.027+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:33.221 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:33.219+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:34.037 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:34.035+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:34.187 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:34.185+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:35.058 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:35.057+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:35.165 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:35.164+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:36.066 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:36.064+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:36.210 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:36.208+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:37.052 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:37.050+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:37.205 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:37.203+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:38.051 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:38.049+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:38.242 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:38.240+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:39.011 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:39.010+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:39.245 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:39.243+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:39.995 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:39.993+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:40.280 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:40.278+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:40.983 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:40.981+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:41.273 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:41.271+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:41.995 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:41.994+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:42.297 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:42.295+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:42.963 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:42.962+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:43.327 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:43.325+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:43.960 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:43.958+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:44.368 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:44.366+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:45.005 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:45.004+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:45.411 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:45.410+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:46.045 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:46.044+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:46.376 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:46.374+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:47.084 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:47.083+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:47.378 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:47.377+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:48.069 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:48.068+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:48.331 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:48.329+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:49.108 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:49.107+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:49.361 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:49.360+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:50.119 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:50.117+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:50.358 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:50.357+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:51.147 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:51.146+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:51.324 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:51.322+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:52.120 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:52.118+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:52.305 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:52.304+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:52.960 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:20:53.162 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:53.161+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:53.272 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:53.270+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:53.299 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:20:53.302 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:20:53.313 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":5989,"stamp":"2023-10-06T00:20:52.152872+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001873"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":1980,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:20:45.410951+0000","last_change":"2023-10-06T00:20:45.410951+0000","last_active":"2023-10-06T00:20:45.410951+0000","last_peered":"2023-10-06T00:20:45.410951+0000","last_clean":"2023-10-06T00:20:45.410951+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:20:45.410951+0000","last_undegraded":"2023-10-06T00:20:45.410951+0000","last_fullsized":"2023-10-06T00:20:45.410951+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1883,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:20:41.272767+0000","last_change":"2023-10-06T00:20:41.272767+0000","last_active":"2023-10-06T00:20:41.272767+0000","last_peered":"2023-10-06T00:20:41.272767+0000","last_clean":"2023-10-06T00:20:41.272767+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:20:41.272767+0000","last_undegraded":"2023-10-06T00:20:41.272767+0000","last_fullsized":"2023-10-06T00:20:41.272767+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8618,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:18:25.215550+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:18:25.215550+0000","last_peered":"2023-10-06T00:18:25.215550+0000","last_clean":"2023-10-06T00:18:25.215550+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:18:25.215550+0000","last_undegraded":"2023-10-06T00:18:25.215550+0000","last_fullsized":"2023-10-06T00:18:25.215550+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 270s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11135,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:20:47.378297+0000","last_change":"2023-10-06T00:20:47.378297+0000","last_active":"2023-10-06T00:20:47.378297+0000","last_peered":"2023-10-06T00:20:47.378297+0000","last_clean":"2023-10-06T00:20:47.378297+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:20:47.378297+0000","last_undegraded":"2023-10-06T00:20:47.378297+0000","last_fullsized":"2023-10-06T00:20:47.378297+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2399,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:20:39.244831+0000","last_change":"2023-10-06T00:20:39.244831+0000","last_active":"2023-10-06T00:20:39.244831+0000","last_peered":"2023-10-06T00:20:39.244831+0000","last_clean":"2023-10-06T00:20:39.244831+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:20:39.244831+0000","last_undegraded":"2023-10-06T00:20:39.244831+0000","last_fullsized":"2023-10-06T00:20:39.244831+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11783,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:20:46.375748+0000","last_change":"2023-10-06T00:20:46.375748+0000","last_active":"2023-10-06T00:20:46.375748+0000","last_peered":"2023-10-06T00:20:46.375748+0000","last_clean":"2023-10-06T00:20:46.375748+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:20:46.375748+0000","last_undegraded":"2023-10-06T00:20:46.375748+0000","last_fullsized":"2023-10-06T00:20:46.375748+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11614,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:18:26.643373+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-06T00:18:26.643373+0000","last_peered":"2023-10-06T00:18:26.643373+0000","last_clean":"2023-10-06T00:18:26.643373+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-06T00:18:26.643373+0000","last_undegraded":"2023-10-06T00:18:26.643373+0000","last_fullsized":"2023-10-06T00:18:26.643373+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1784,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:18:27.028238+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-06T00:18:27.028238+0000","last_peered":"2023-10-06T00:18:27.028238+0000","last_clean":"2023-10-06T00:18:27.028238+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-06T00:18:27.028238+0000","last_undegraded":"2023-10-06T00:18:27.028238+0000","last_fullsized":"2023-10-06T00:18:27.028238+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1804,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:20:37.204202+0000","last_change":"2023-10-06T00:20:37.204202+0000","last_active":"2023-10-06T00:20:37.204202+0000","last_peered":"2023-10-06T00:20:37.204202+0000","last_clean":"2023-10-06T00:20:37.204202+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:20:37.204202+0000","last_undegraded":"2023-10-06T00:20:37.204202+0000","last_fullsized":"2023-10-06T00:20:37.204202+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740711,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.20100000000000001,"5min":0.22500000000000001,"15min":0.22},"min":{"1min":0.13800000000000001,"5min":0.13800000000000001,"15min":0.13800000000000001},"max":{"1min":0.52100000000000002,"5min":1.153,"15min":1.153},"last":0.16700000000000001},{"interface":"front","average":{"1min":0.27300000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"min":{"1min":0.17999999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.20599999999999999}]},{"osd":1,"last update":"Fri Oct 6 00:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.253,"5min":0.255,"15min":0.248},"min":{"1min":0.187,"5min":0.157,"15min":0.157},"max":{"1min":0.35299999999999998,"5min":0.44500000000000001,"15min":0.48699999999999999},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.312,"5min":0.28999999999999998,"15min":0.27700000000000002},"min":{"1min":0.223,"5min":0.218,"15min":0.19600000000000001},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.25}]}]},{"osd":0,"up_from":8,"seq":34359740711,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.193,"5min":0.19900000000000001,"15min":0.19800000000000001},"min":{"1min":0.14899999999999999,"5min":0.14399999999999999,"15min":0.128},"max":{"1min":0.29499999999999998,"5min":0.34999999999999998,"15min":0.41299999999999998},"last":0.248},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.26500000000000001,"15min":0.26400000000000001},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.159},"max":{"1min":0.312,"5min":0.443,"15min":0.443},"last":0.26800000000000002}]},{"osd":2,"last update":"Fri Oct 6 00:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.22800000000000001,"5min":0.23499999999999999,"15min":0.23300000000000001},"min":{"1min":0.20000000000000001,"5min":0.17100000000000001,"15min":0.16300000000000001},"max":{"1min":0.27800000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"min":{"1min":0.24199999999999999,"5min":0.215,"15min":0.21299999999999999},"max":{"1min":0.34999999999999998,"5min":0.47499999999999998,"15min":0.53600000000000003},"last":0.30099999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740706,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:20:36 2023","interfaces":[{"interface":"back","average":{"1min":0.20300000000000001,"5min":0.218,"15min":0.21099999999999999},"min":{"1min":0.14499999999999999,"5min":0.13100000000000001,"15min":0.13100000000000001},"max":{"1min":0.371,"5min":0.41399999999999998,"15min":0.497},"last":0.17000000000000001},{"interface":"front","average":{"1min":0.222,"5min":0.25,"15min":0.26200000000000001},"min":{"1min":0.17199999999999999,"5min":0.16300000000000001,"15min":0.154},"max":{"1min":0.315,"5min":0.47599999999999998,"15min":0.51100000000000001},"last":0.23599999999999999}]},{"osd":2,"last update":"Fri Oct 6 00:20:36 2023","interfaces":[{"interface":"back","average":{"1min":0.24099999999999999,"5min":0.23999999999999999,"15min":0.23400000000000001},"min":{"1min":0.19600000000000001,"5min":0.17599999999999999,"15min":0.17100000000000001},"max":{"1min":0.33100000000000002,"5min":0.44900000000000001,"15min":0.46999999999999997},"last":0.20899999999999999},{"interface":"front","average":{"1min":0.24199999999999999,"5min":0.248,"15min":0.26400000000000001},"min":{"1min":0.188,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.35399999999999998,"5min":0.50900000000000001,"15min":0.54700000000000004},"last":0.222}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:20:53.315 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:20:53.315 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:20:53.315 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:20:53.316 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:20:53.316 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:20:53.316 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:20:53.316 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:20:53.316 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:20:53.317 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:20:53.317 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:20:54.144 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:54.142+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:54.262 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:54.260+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:55.139 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:55.138+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:55.274 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:55.273+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:56.149 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:56.148+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:56.243 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:56.242+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:57.118 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:57.116+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:57.241 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:57.240+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:58.124 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:58.124+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:58.195 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:58.195+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:59.111 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:20:59.111+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:20:59.210 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:20:59.210+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:00.083 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:00.082+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:00.195 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:00.195+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:01.062 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:01.061+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:01.152 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:01.151+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:02.054 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:02.054+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:02.141 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:02.140+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:03.058 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:03.058+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:03.123 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:03.122+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:04.032 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:04.032+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:04.080 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:04.080+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:05.041 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:05.041+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:05.076 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:05.076+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:06.051 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:06.050+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:06.076 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:06.076+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:07.068 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:07.068+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:07.076 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:07.075+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:08.041 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:08.040+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:08.075 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:08.075+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:09.088 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:09.088+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:09.098 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:09.098+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:10.075 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:10.074+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:10.115 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:10.115+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:11.079 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:11.078+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:11.123 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:11.123+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:12.043 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:12.042+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:12.100 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:12.099+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:13.068 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:13.067+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:13.110 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:13.110+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:13.318 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:21:13.688 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:21:13.689 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:21:13.701 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":5999,"stamp":"2023-10-06T00:21:12.155951+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001758"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":1983,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:20:56.243123+0000","last_change":"2023-10-06T00:20:56.243123+0000","last_active":"2023-10-06T00:20:56.243123+0000","last_peered":"2023-10-06T00:20:56.243123+0000","last_clean":"2023-10-06T00:20:56.243123+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:20:56.243123+0000","last_undegraded":"2023-10-06T00:20:56.243123+0000","last_fullsized":"2023-10-06T00:20:56.243123+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1889,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:21:05.076450+0000","last_change":"2023-10-06T00:21:05.076450+0000","last_active":"2023-10-06T00:21:05.076450+0000","last_peered":"2023-10-06T00:21:05.076450+0000","last_clean":"2023-10-06T00:21:05.076450+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:21:05.076450+0000","last_undegraded":"2023-10-06T00:21:05.076450+0000","last_fullsized":"2023-10-06T00:21:05.076450+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8618,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:18:25.215550+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:18:25.215550+0000","last_peered":"2023-10-06T00:18:25.215550+0000","last_clean":"2023-10-06T00:18:25.215550+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:18:25.215550+0000","last_undegraded":"2023-10-06T00:18:25.215550+0000","last_fullsized":"2023-10-06T00:18:25.215550+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 270s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11138,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:21:00.195419+0000","last_change":"2023-10-06T00:21:00.195419+0000","last_active":"2023-10-06T00:21:00.195419+0000","last_peered":"2023-10-06T00:21:00.195419+0000","last_clean":"2023-10-06T00:21:00.195419+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:21:00.195419+0000","last_undegraded":"2023-10-06T00:21:00.195419+0000","last_fullsized":"2023-10-06T00:21:00.195419+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2405,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:21:03.122594+0000","last_change":"2023-10-06T00:21:03.122594+0000","last_active":"2023-10-06T00:21:03.122594+0000","last_peered":"2023-10-06T00:21:03.122594+0000","last_clean":"2023-10-06T00:21:03.122594+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:21:03.122594+0000","last_undegraded":"2023-10-06T00:21:03.122594+0000","last_fullsized":"2023-10-06T00:21:03.122594+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11786,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:20:58.195219+0000","last_change":"2023-10-06T00:20:58.195219+0000","last_active":"2023-10-06T00:20:58.195219+0000","last_peered":"2023-10-06T00:20:58.195219+0000","last_clean":"2023-10-06T00:20:58.195219+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:20:58.195219+0000","last_undegraded":"2023-10-06T00:20:58.195219+0000","last_fullsized":"2023-10-06T00:20:58.195219+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11614,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:18:26.643373+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-06T00:18:26.643373+0000","last_peered":"2023-10-06T00:18:26.643373+0000","last_clean":"2023-10-06T00:18:26.643373+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-06T00:18:26.643373+0000","last_undegraded":"2023-10-06T00:18:26.643373+0000","last_fullsized":"2023-10-06T00:18:26.643373+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1784,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:18:27.028238+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-06T00:18:27.028238+0000","last_peered":"2023-10-06T00:18:27.028238+0000","last_clean":"2023-10-06T00:18:27.028238+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-06T00:18:27.028238+0000","last_undegraded":"2023-10-06T00:18:27.028238+0000","last_fullsized":"2023-10-06T00:18:27.028238+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1810,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:21:01.151748+0000","last_change":"2023-10-06T00:21:01.151748+0000","last_active":"2023-10-06T00:21:01.151748+0000","last_peered":"2023-10-06T00:21:01.151748+0000","last_clean":"2023-10-06T00:21:01.151748+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:21:01.151748+0000","last_undegraded":"2023-10-06T00:21:01.151748+0000","last_fullsized":"2023-10-06T00:21:01.151748+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740715,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.20100000000000001,"5min":0.22500000000000001,"15min":0.22},"min":{"1min":0.13800000000000001,"5min":0.13800000000000001,"15min":0.13800000000000001},"max":{"1min":0.52100000000000002,"5min":1.153,"15min":1.153},"last":0.20200000000000001},{"interface":"front","average":{"1min":0.27300000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"min":{"1min":0.17999999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.248}]},{"osd":1,"last update":"Fri Oct 6 00:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.253,"5min":0.255,"15min":0.248},"min":{"1min":0.187,"5min":0.157,"15min":0.157},"max":{"1min":0.35299999999999998,"5min":0.44500000000000001,"15min":0.48699999999999999},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.312,"5min":0.28999999999999998,"15min":0.27700000000000002},"min":{"1min":0.223,"5min":0.218,"15min":0.19600000000000001},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.216}]}]},{"osd":0,"up_from":8,"seq":34359740715,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.193,"5min":0.19900000000000001,"15min":0.19800000000000001},"min":{"1min":0.14899999999999999,"5min":0.14399999999999999,"15min":0.128},"max":{"1min":0.29499999999999998,"5min":0.34999999999999998,"15min":0.41299999999999998},"last":0.13900000000000001},{"interface":"front","average":{"1min":0.26300000000000001,"5min":0.26500000000000001,"15min":0.26400000000000001},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.159},"max":{"1min":0.312,"5min":0.443,"15min":0.443},"last":0.19800000000000001}]},{"osd":2,"last update":"Fri Oct 6 00:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.22800000000000001,"5min":0.23499999999999999,"15min":0.23300000000000001},"min":{"1min":0.20000000000000001,"5min":0.17100000000000001,"15min":0.16300000000000001},"max":{"1min":0.27800000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"last":0.16600000000000001},{"interface":"front","average":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"min":{"1min":0.24199999999999999,"5min":0.215,"15min":0.21299999999999999},"max":{"1min":0.34999999999999998,"5min":0.47499999999999998,"15min":0.53600000000000003},"last":0.221}]}]},{"osd":1,"up_from":8,"seq":34359740710,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:20:36 2023","interfaces":[{"interface":"back","average":{"1min":0.20300000000000001,"5min":0.218,"15min":0.21099999999999999},"min":{"1min":0.14499999999999999,"5min":0.13100000000000001,"15min":0.13100000000000001},"max":{"1min":0.371,"5min":0.41399999999999998,"15min":0.497},"last":0.17599999999999999},{"interface":"front","average":{"1min":0.222,"5min":0.25,"15min":0.26200000000000001},"min":{"1min":0.17199999999999999,"5min":0.16300000000000001,"15min":0.154},"max":{"1min":0.315,"5min":0.47599999999999998,"15min":0.51100000000000001},"last":0.22}]},{"osd":2,"last update":"Fri Oct 6 00:20:36 2023","interfaces":[{"interface":"back","average":{"1min":0.24099999999999999,"5min":0.23999999999999999,"15min":0.23400000000000001},"min":{"1min":0.19600000000000001,"5min":0.17599999999999999,"15min":0.17100000000000001},"max":{"1min":0.33100000000000002,"5min":0.44900000000000001,"15min":0.46999999999999997},"last":0.20899999999999999},{"interface":"front","average":{"1min":0.24199999999999999,"5min":0.248,"15min":0.26400000000000001},"min":{"1min":0.188,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.35399999999999998,"5min":0.50900000000000001,"15min":0.54700000000000004},"last":0.23400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:21:13.703 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:21:13.703 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:21:13.703 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:21:13.704 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:21:13.704 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:21:13.704 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:21:13.704 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:21:13.705 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:21:13.705 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:21:13.705 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:21:14.044 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:14.044+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:14.142 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:14.142+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:15.042 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:15.041+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:15.160 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:15.160+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:16.006 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:16.005+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:16.192 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:16.191+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:16.961 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:16.960+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:17.149 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:17.149+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:17.974 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:17.974+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:18.128 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:18.128+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:18.928 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:18.927+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:19.178 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:19.176+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:19.914 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:19.913+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:20.149 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:20.147+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:20.932 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:20.931+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:21.195 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:21.194+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:21.981 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:21.979+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:22.178 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:22.174+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:22.968 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:22.966+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:23.223 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:23.221+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:23.933 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:23.932+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:24.179 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:24.178+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:24.904 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:24.902+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:25.168 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:25.166+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:25.930 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:25.929+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:26.172 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:26.170+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:26.940 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:26.938+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:27.207 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:27.206+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:27.934 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:27.932+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:28.202 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:28.200+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:28.949 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:28.947+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:29.176 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:29.175+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:29.983 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:29.981+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:30.154 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:30.152+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:30.973 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:30.971+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:31.124 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:31.123+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:31.952 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:31.950+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:32.091 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:32.090+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:32.966 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:32.964+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:33.044 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:33.043+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:33.706 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:21:33.935 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:33.933+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:34.060 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:21:34.061 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:21:34.073 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":6009,"stamp":"2023-10-06T00:21:32.158997+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001788"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":1989,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:21:25.167449+0000","last_change":"2023-10-06T00:21:25.167449+0000","last_active":"2023-10-06T00:21:25.167449+0000","last_peered":"2023-10-06T00:21:25.167449+0000","last_clean":"2023-10-06T00:21:25.167449+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:21:25.167449+0000","last_undegraded":"2023-10-06T00:21:25.167449+0000","last_fullsized":"2023-10-06T00:21:25.167449+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1892,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:21:20.148870+0000","last_change":"2023-10-06T00:21:20.148870+0000","last_active":"2023-10-06T00:21:20.148870+0000","last_peered":"2023-10-06T00:21:20.148870+0000","last_clean":"2023-10-06T00:21:20.148870+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:21:20.148870+0000","last_undegraded":"2023-10-06T00:21:20.148870+0000","last_fullsized":"2023-10-06T00:21:20.148870+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8618,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:18:25.215550+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:18:25.215550+0000","last_peered":"2023-10-06T00:18:25.215550+0000","last_clean":"2023-10-06T00:18:25.215550+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:18:25.215550+0000","last_undegraded":"2023-10-06T00:18:25.215550+0000","last_fullsized":"2023-10-06T00:18:25.215550+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 270s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11144,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:21:28.201546+0000","last_change":"2023-10-06T00:21:28.201546+0000","last_active":"2023-10-06T00:21:28.201546+0000","last_peered":"2023-10-06T00:21:28.201546+0000","last_clean":"2023-10-06T00:21:28.201546+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:21:28.201546+0000","last_undegraded":"2023-10-06T00:21:28.201546+0000","last_fullsized":"2023-10-06T00:21:28.201546+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2408,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:21:19.177628+0000","last_change":"2023-10-06T00:21:19.177628+0000","last_active":"2023-10-06T00:21:19.177628+0000","last_peered":"2023-10-06T00:21:19.177628+0000","last_clean":"2023-10-06T00:21:19.177628+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:21:19.177628+0000","last_undegraded":"2023-10-06T00:21:19.177628+0000","last_fullsized":"2023-10-06T00:21:19.177628+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11792,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:21:26.171655+0000","last_change":"2023-10-06T00:21:26.171655+0000","last_active":"2023-10-06T00:21:26.171655+0000","last_peered":"2023-10-06T00:21:26.171655+0000","last_clean":"2023-10-06T00:21:26.171655+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:21:26.171655+0000","last_undegraded":"2023-10-06T00:21:26.171655+0000","last_fullsized":"2023-10-06T00:21:26.171655+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11614,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:18:26.643373+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-06T00:18:26.643373+0000","last_peered":"2023-10-06T00:18:26.643373+0000","last_clean":"2023-10-06T00:18:26.643373+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-06T00:18:26.643373+0000","last_undegraded":"2023-10-06T00:18:26.643373+0000","last_fullsized":"2023-10-06T00:18:26.643373+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1784,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:18:27.028238+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-06T00:18:27.028238+0000","last_peered":"2023-10-06T00:18:27.028238+0000","last_clean":"2023-10-06T00:18:27.028238+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-06T00:18:27.028238+0000","last_undegraded":"2023-10-06T00:18:27.028238+0000","last_fullsized":"2023-10-06T00:18:27.028238+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1813,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:21:18.128209+0000","last_change":"2023-10-06T00:21:18.128209+0000","last_active":"2023-10-06T00:21:18.128209+0000","last_peered":"2023-10-06T00:21:18.128209+0000","last_clean":"2023-10-06T00:21:18.128209+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:21:18.128209+0000","last_undegraded":"2023-10-06T00:21:18.128209+0000","last_fullsized":"2023-10-06T00:21:18.128209+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740719,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.191,"5min":0.20799999999999999,"15min":0.218},"min":{"1min":0.152,"5min":0.13800000000000001,"15min":0.13800000000000001},"max":{"1min":0.23100000000000001,"5min":0.52100000000000002,"15min":1.153},"last":0.185},{"interface":"front","average":{"1min":0.22800000000000001,"5min":0.23899999999999999,"15min":0.24099999999999999},"min":{"1min":0.188,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.29299999999999998,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.22500000000000001}]},{"osd":1,"last update":"Fri Oct 6 00:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.249,"5min":0.252,"15min":0.248},"min":{"1min":0.20499999999999999,"5min":0.157,"15min":0.157},"max":{"1min":0.316,"5min":0.44500000000000001,"15min":0.46700000000000003},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.25,"5min":0.28100000000000003,"15min":0.27500000000000002},"min":{"1min":0.17999999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.31900000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.26500000000000001}]}]},{"osd":0,"up_from":8,"seq":34359740719,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.222,"5min":0.20000000000000001,"15min":0.19800000000000001},"min":{"1min":0.13900000000000001,"5min":0.13900000000000001,"15min":0.128},"max":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"last":0.19},{"interface":"front","average":{"1min":0.28000000000000003,"5min":0.26400000000000001,"15min":0.26500000000000001},"min":{"1min":0.19700000000000001,"5min":0.17799999999999999,"15min":0.159},"max":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.443},"last":0.25900000000000001}]},{"osd":2,"last update":"Fri Oct 6 00:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.255,"5min":0.23599999999999999,"15min":0.23400000000000001},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.16300000000000001},"max":{"1min":0.438,"5min":0.438,"15min":0.438},"last":0.223},{"interface":"front","average":{"1min":0.315,"5min":0.29599999999999999,"15min":0.29599999999999999},"min":{"1min":0.221,"5min":0.215,"15min":0.21299999999999999},"max":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.47499999999999998},"last":0.29599999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740714,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:20:36 2023","interfaces":[{"interface":"back","average":{"1min":0.20300000000000001,"5min":0.218,"15min":0.21099999999999999},"min":{"1min":0.14499999999999999,"5min":0.13100000000000001,"15min":0.13100000000000001},"max":{"1min":0.371,"5min":0.41399999999999998,"15min":0.497},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.222,"5min":0.25,"15min":0.26200000000000001},"min":{"1min":0.17199999999999999,"5min":0.16300000000000001,"15min":0.154},"max":{"1min":0.315,"5min":0.47599999999999998,"15min":0.51100000000000001},"last":0.24399999999999999}]},{"osd":2,"last update":"Fri Oct 6 00:20:36 2023","interfaces":[{"interface":"back","average":{"1min":0.24099999999999999,"5min":0.23999999999999999,"15min":0.23400000000000001},"min":{"1min":0.19600000000000001,"5min":0.17599999999999999,"15min":0.17100000000000001},"max":{"1min":0.33100000000000002,"5min":0.44900000000000001,"15min":0.46999999999999997},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.24199999999999999,"5min":0.248,"15min":0.26400000000000001},"min":{"1min":0.188,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.35399999999999998,"5min":0.50900000000000001,"15min":0.54700000000000004},"last":0.26400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:21:34.075 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:21:34.075 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:21:34.075 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:21:34.075 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:21:34.076 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:21:34.076 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:21:34.076 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:21:34.076 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:21:34.077 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:21:34.077 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:21:34.092 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:34.090+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:34.950 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:34.948+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:35.130 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:35.128+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:35.917 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:35.916+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:36.116 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:36.114+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:36.914 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:36.913+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:37.126 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:37.125+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:37.961 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:37.959+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:38.161 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:38.160+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:38.957 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:38.955+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:39.184 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:39.182+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:39.965 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:39.963+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:40.183 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:40.181+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:40.959 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:40.958+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:41.208 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:41.207+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:41.990 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:41.988+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:42.175 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:42.173+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:43.020 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:43.018+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:43.133 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:43.131+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:43.982 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:43.980+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:44.138 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:44.137+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:45.013 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:45.011+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:45.144 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:45.143+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:46.000 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:45.998+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:46.126 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:46.125+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:47.028 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:47.027+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:47.142 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:47.141+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:48.008 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:48.006+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:48.191 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:48.189+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:48.990 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:48.988+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:49.176 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:49.174+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:49.958 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:49.957+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:50.184 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:50.183+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:50.940 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:50.938+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:51.199 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:51.198+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:51.905 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:51.904+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:52.220 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:52.219+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:52.917 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:52.916+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:53.192 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:53.191+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:53.891 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:53.889+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:54.077 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:21:54.163 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:54.161+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:54.420 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:21:54.422 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:21:54.434 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":6020,"stamp":"2023-10-06T00:21:54.162480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001935"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":1995,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:21:45.144051+0000","last_change":"2023-10-06T00:21:45.144051+0000","last_active":"2023-10-06T00:21:45.144051+0000","last_peered":"2023-10-06T00:21:45.144051+0000","last_clean":"2023-10-06T00:21:45.144051+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:21:45.144051+0000","last_undegraded":"2023-10-06T00:21:45.144051+0000","last_fullsized":"2023-10-06T00:21:45.144051+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1898,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:21:41.208289+0000","last_change":"2023-10-06T00:21:41.208289+0000","last_active":"2023-10-06T00:21:41.208289+0000","last_peered":"2023-10-06T00:21:41.208289+0000","last_clean":"2023-10-06T00:21:41.208289+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:21:41.208289+0000","last_undegraded":"2023-10-06T00:21:41.208289+0000","last_fullsized":"2023-10-06T00:21:41.208289+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8618,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:18:25.215550+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:18:25.215550+0000","last_peered":"2023-10-06T00:18:25.215550+0000","last_clean":"2023-10-06T00:18:25.215550+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:18:25.215550+0000","last_undegraded":"2023-10-06T00:18:25.215550+0000","last_fullsized":"2023-10-06T00:18:25.215550+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 270s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11150,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:21:50.184230+0000","last_change":"2023-10-06T00:21:50.184230+0000","last_active":"2023-10-06T00:21:50.184230+0000","last_peered":"2023-10-06T00:21:50.184230+0000","last_clean":"2023-10-06T00:21:50.184230+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:21:50.184230+0000","last_undegraded":"2023-10-06T00:21:50.184230+0000","last_fullsized":"2023-10-06T00:21:50.184230+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2414,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:21:40.182666+0000","last_change":"2023-10-06T00:21:40.182666+0000","last_active":"2023-10-06T00:21:40.182666+0000","last_peered":"2023-10-06T00:21:40.182666+0000","last_clean":"2023-10-06T00:21:40.182666+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:21:40.182666+0000","last_undegraded":"2023-10-06T00:21:40.182666+0000","last_fullsized":"2023-10-06T00:21:40.182666+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11798,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:21:46.125973+0000","last_change":"2023-10-06T00:21:46.125973+0000","last_active":"2023-10-06T00:21:46.125973+0000","last_peered":"2023-10-06T00:21:46.125973+0000","last_clean":"2023-10-06T00:21:46.125973+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:21:46.125973+0000","last_undegraded":"2023-10-06T00:21:46.125973+0000","last_fullsized":"2023-10-06T00:21:46.125973+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11614,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:18:26.643373+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-06T00:18:26.643373+0000","last_peered":"2023-10-06T00:18:26.643373+0000","last_clean":"2023-10-06T00:18:26.643373+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-06T00:18:26.643373+0000","last_undegraded":"2023-10-06T00:18:26.643373+0000","last_fullsized":"2023-10-06T00:18:26.643373+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1784,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:18:27.028238+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-06T00:18:27.028238+0000","last_peered":"2023-10-06T00:18:27.028238+0000","last_clean":"2023-10-06T00:18:27.028238+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-06T00:18:27.028238+0000","last_undegraded":"2023-10-06T00:18:27.028238+0000","last_fullsized":"2023-10-06T00:18:27.028238+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1822,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:21:51.198848+0000","last_change":"2023-10-06T00:21:51.198848+0000","last_active":"2023-10-06T00:21:51.198848+0000","last_peered":"2023-10-06T00:21:51.198848+0000","last_clean":"2023-10-06T00:21:51.198848+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:21:51.198848+0000","last_undegraded":"2023-10-06T00:21:51.198848+0000","last_fullsized":"2023-10-06T00:21:51.198848+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740723,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.191,"5min":0.20799999999999999,"15min":0.218},"min":{"1min":0.152,"5min":0.13800000000000001,"15min":0.13800000000000001},"max":{"1min":0.23100000000000001,"5min":0.52100000000000002,"15min":1.153},"last":0.17000000000000001},{"interface":"front","average":{"1min":0.22800000000000001,"5min":0.23899999999999999,"15min":0.24099999999999999},"min":{"1min":0.188,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.29299999999999998,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.23000000000000001}]},{"osd":1,"last update":"Fri Oct 6 00:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.249,"5min":0.252,"15min":0.248},"min":{"1min":0.20499999999999999,"5min":0.157,"15min":0.157},"max":{"1min":0.316,"5min":0.44500000000000001,"15min":0.46700000000000003},"last":0.21099999999999999},{"interface":"front","average":{"1min":0.25,"5min":0.28100000000000003,"15min":0.27500000000000002},"min":{"1min":0.17999999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.31900000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.188}]}]},{"osd":0,"up_from":8,"seq":34359740724,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.222,"5min":0.20000000000000001,"15min":0.19800000000000001},"min":{"1min":0.13900000000000001,"5min":0.13900000000000001,"15min":0.128},"max":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"last":0.17199999999999999},{"interface":"front","average":{"1min":0.28000000000000003,"5min":0.26400000000000001,"15min":0.26500000000000001},"min":{"1min":0.19700000000000001,"5min":0.17799999999999999,"15min":0.159},"max":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.443},"last":0.24399999999999999}]},{"osd":2,"last update":"Fri Oct 6 00:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.255,"5min":0.23599999999999999,"15min":0.23400000000000001},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.16300000000000001},"max":{"1min":0.438,"5min":0.438,"15min":0.438},"last":0.219},{"interface":"front","average":{"1min":0.315,"5min":0.29599999999999999,"15min":0.29599999999999999},"min":{"1min":0.221,"5min":0.215,"15min":0.21299999999999999},"max":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.47499999999999998},"last":0.26700000000000002}]}]},{"osd":1,"up_from":8,"seq":34359740719,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:21:38 2023","interfaces":[{"interface":"back","average":{"1min":0.20499999999999999,"5min":0.21099999999999999,"15min":0.21199999999999999},"min":{"1min":0.161,"5min":0.13100000000000001,"15min":0.13100000000000001},"max":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.497},"last":0.221},{"interface":"front","average":{"1min":0.23599999999999999,"5min":0.24199999999999999,"15min":0.26000000000000001},"min":{"1min":0.17799999999999999,"5min":0.16300000000000001,"15min":0.154},"max":{"1min":0.40400000000000003,"5min":0.41199999999999998,"15min":0.51100000000000001},"last":0.248}]},{"osd":2,"last update":"Fri Oct 6 00:21:38 2023","interfaces":[{"interface":"back","average":{"1min":0.24199999999999999,"5min":0.23799999999999999,"15min":0.23400000000000001},"min":{"1min":0.19700000000000001,"5min":0.18099999999999999,"15min":0.17100000000000001},"max":{"1min":0.434,"5min":0.44900000000000001,"15min":0.46999999999999997},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.25700000000000001,"5min":0.24299999999999999,"15min":0.26200000000000001},"min":{"1min":0.20399999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.54700000000000004},"last":0.26500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:21:54.436 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:21:54.436 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:21:54.436 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:21:54.437 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:21:54.437 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:21:54.437 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:21:54.438 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:21:54.438 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:21:54.438 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:21:54.438 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:21:54.928 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:54.927+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:55.143 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:55.142+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:55.951 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:55.949+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:56.173 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:56.171+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:56.969 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:56.968+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:57.159 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:57.158+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:57.995 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:57.995+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:58.121 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:58.120+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:58.962 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:58.962+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:59.153 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:21:59.152+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:21:59.960 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:21:59.959+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:00.122 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:00.122+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:00.914 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:00.914+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:01.089 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:01.089+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:01.945 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:01.944+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:02.082 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:02.082+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:02.912 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:02.912+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:03.036 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:03.036+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:03.915 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:03.915+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:04.003 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:04.003+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:04.877 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:04.876+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:04.959 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:04.958+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:05.870 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:05.870+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:05.939 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:05.938+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:06.832 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:06.832+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:06.981 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:06.980+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:07.871 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:07.870+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:07.973 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:07.973+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:08.911 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:08.910+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:08.986 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:08.985+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:09.903 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:09.902+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:09.960 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:09.960+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:10.902 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:10.902+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:10.928 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:10.928+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:11.856 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:11.855+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:11.934 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:11.934+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:12.847 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:12.846+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:12.940 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:12.940+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:13.803 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:13.802+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:13.926 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:13.925+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:14.439 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:22:14.816 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:14.816+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:14.824 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:22:14.827 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:22:14.839 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":6030,"stamp":"2023-10-06T00:22:14.166014+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002058"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":1998,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:22:05.938293+0000","last_change":"2023-10-06T00:22:05.938293+0000","last_active":"2023-10-06T00:22:05.938293+0000","last_peered":"2023-10-06T00:22:05.938293+0000","last_clean":"2023-10-06T00:22:05.938293+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:22:05.938293+0000","last_undegraded":"2023-10-06T00:22:05.938293+0000","last_fullsized":"2023-10-06T00:22:05.938293+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1901,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:22:04.958752+0000","last_change":"2023-10-06T00:22:04.958752+0000","last_active":"2023-10-06T00:22:04.958752+0000","last_peered":"2023-10-06T00:22:04.958752+0000","last_clean":"2023-10-06T00:22:04.958752+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:22:04.958752+0000","last_undegraded":"2023-10-06T00:22:04.958752+0000","last_fullsized":"2023-10-06T00:22:04.958752+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8618,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:18:25.215550+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:18:25.215550+0000","last_peered":"2023-10-06T00:18:25.215550+0000","last_clean":"2023-10-06T00:18:25.215550+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:18:25.215550+0000","last_undegraded":"2023-10-06T00:18:25.215550+0000","last_fullsized":"2023-10-06T00:18:25.215550+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 270s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11153,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:22:07.973445+0000","last_change":"2023-10-06T00:22:07.973445+0000","last_active":"2023-10-06T00:22:07.973445+0000","last_peered":"2023-10-06T00:22:07.973445+0000","last_clean":"2023-10-06T00:22:07.973445+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:22:07.973445+0000","last_undegraded":"2023-10-06T00:22:07.973445+0000","last_fullsized":"2023-10-06T00:22:07.973445+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2417,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:21:56.172948+0000","last_change":"2023-10-06T00:21:56.172948+0000","last_active":"2023-10-06T00:21:56.172948+0000","last_peered":"2023-10-06T00:21:56.172948+0000","last_clean":"2023-10-06T00:21:56.172948+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:21:56.172948+0000","last_undegraded":"2023-10-06T00:21:56.172948+0000","last_fullsized":"2023-10-06T00:21:56.172948+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11801,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:22:06.980844+0000","last_change":"2023-10-06T00:22:06.980844+0000","last_active":"2023-10-06T00:22:06.980844+0000","last_peered":"2023-10-06T00:22:06.980844+0000","last_clean":"2023-10-06T00:22:06.980844+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:22:06.980844+0000","last_undegraded":"2023-10-06T00:22:06.980844+0000","last_fullsized":"2023-10-06T00:22:06.980844+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11614,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:18:26.643373+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-06T00:18:26.643373+0000","last_peered":"2023-10-06T00:18:26.643373+0000","last_clean":"2023-10-06T00:18:26.643373+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-06T00:18:26.643373+0000","last_undegraded":"2023-10-06T00:18:26.643373+0000","last_fullsized":"2023-10-06T00:18:26.643373+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1784,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:18:27.028238+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-06T00:18:27.028238+0000","last_peered":"2023-10-06T00:18:27.028238+0000","last_clean":"2023-10-06T00:18:27.028238+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-06T00:18:27.028238+0000","last_undegraded":"2023-10-06T00:18:27.028238+0000","last_fullsized":"2023-10-06T00:18:27.028238+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1825,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:22:09.960041+0000","last_change":"2023-10-06T00:22:09.960041+0000","last_active":"2023-10-06T00:22:09.960041+0000","last_peered":"2023-10-06T00:22:09.960041+0000","last_clean":"2023-10-06T00:22:09.960041+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:22:09.960041+0000","last_undegraded":"2023-10-06T00:22:09.960041+0000","last_fullsized":"2023-10-06T00:22:09.960041+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740727,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.191,"5min":0.20799999999999999,"15min":0.218},"min":{"1min":0.152,"5min":0.13800000000000001,"15min":0.13800000000000001},"max":{"1min":0.23100000000000001,"5min":0.52100000000000002,"15min":1.153},"last":0.215},{"interface":"front","average":{"1min":0.22800000000000001,"5min":0.23899999999999999,"15min":0.24099999999999999},"min":{"1min":0.188,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.29299999999999998,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.252}]},{"osd":1,"last update":"Fri Oct 6 00:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.249,"5min":0.252,"15min":0.248},"min":{"1min":0.20499999999999999,"5min":0.157,"15min":0.157},"max":{"1min":0.316,"5min":0.44500000000000001,"15min":0.46700000000000003},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.25,"5min":0.28100000000000003,"15min":0.27500000000000002},"min":{"1min":0.17999999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.31900000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.28799999999999998}]}]},{"osd":0,"up_from":8,"seq":34359740728,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.222,"5min":0.20000000000000001,"15min":0.19800000000000001},"min":{"1min":0.13900000000000001,"5min":0.13900000000000001,"15min":0.128},"max":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"last":0.183},{"interface":"front","average":{"1min":0.28000000000000003,"5min":0.26400000000000001,"15min":0.26500000000000001},"min":{"1min":0.19700000000000001,"5min":0.17799999999999999,"15min":0.159},"max":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.443},"last":0.28799999999999998}]},{"osd":2,"last update":"Fri Oct 6 00:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.255,"5min":0.23599999999999999,"15min":0.23400000000000001},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.16300000000000001},"max":{"1min":0.438,"5min":0.438,"15min":0.438},"last":0.251},{"interface":"front","average":{"1min":0.315,"5min":0.29599999999999999,"15min":0.29599999999999999},"min":{"1min":0.221,"5min":0.215,"15min":0.21299999999999999},"max":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.47499999999999998},"last":0.32000000000000001}]}]},{"osd":1,"up_from":8,"seq":34359740723,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:21:38 2023","interfaces":[{"interface":"back","average":{"1min":0.20499999999999999,"5min":0.21099999999999999,"15min":0.21199999999999999},"min":{"1min":0.161,"5min":0.13100000000000001,"15min":0.13100000000000001},"max":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.497},"last":0.20899999999999999},{"interface":"front","average":{"1min":0.23599999999999999,"5min":0.24199999999999999,"15min":0.26000000000000001},"min":{"1min":0.17799999999999999,"5min":0.16300000000000001,"15min":0.154},"max":{"1min":0.40400000000000003,"5min":0.41199999999999998,"15min":0.51100000000000001},"last":0.23999999999999999}]},{"osd":2,"last update":"Fri Oct 6 00:21:38 2023","interfaces":[{"interface":"back","average":{"1min":0.24199999999999999,"5min":0.23799999999999999,"15min":0.23400000000000001},"min":{"1min":0.19700000000000001,"5min":0.18099999999999999,"15min":0.17100000000000001},"max":{"1min":0.434,"5min":0.44900000000000001,"15min":0.46999999999999997},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.25700000000000001,"5min":0.24299999999999999,"15min":0.26200000000000001},"min":{"1min":0.20399999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.54700000000000004},"last":0.254}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:22:14.841 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:22:14.841 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:22:14.841 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:22:14.842 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:22:14.842 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:22:14.842 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:22:14.843 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:22:14.843 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:22:14.843 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:22:14.844 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:22:14.961 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:14.960+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:15.832 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:15.832+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:15.971 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:15.970+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:16.870 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:16.869+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:17.020 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:17.018+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:17.859 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:17.857+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:18.064 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:18.063+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:18.882 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:18.880+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:19.073 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:19.072+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:19.890 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:19.889+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:20.052 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:20.051+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:20.882 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:20.880+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:21.013 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:21.011+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:21.870 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:21.869+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:22.000 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:21.999+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:22.842 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:22.841+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:23.008 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:23.007+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:23.874 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:23.873+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:24.057 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:24.056+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:24.890 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:24.889+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:25.072 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:25.070+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:25.850 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:25.849+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:26.047 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:26.045+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:26.840 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:26.839+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:27.085 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:27.083+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:27.881 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:27.880+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:28.087 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:28.086+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:28.898 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:28.897+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:29.090 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:29.088+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:29.858 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:29.857+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:30.062 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:30.060+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:30.870 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:30.869+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:31.109 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:31.108+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:31.866 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:31.865+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:32.070 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:32.069+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:32.835 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:32.833+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:33.089 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:33.087+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:33.795 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:33.793+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:34.045 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:34.043+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:34.820 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:34.819+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:34.845 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:22:35.068 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:35.066+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:35.185 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:22:35.187 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:22:35.199 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":6040,"stamp":"2023-10-06T00:22:34.169282+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001783"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":2001,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:22:19.073404+0000","last_change":"2023-10-06T00:22:19.073404+0000","last_active":"2023-10-06T00:22:19.073404+0000","last_peered":"2023-10-06T00:22:19.073404+0000","last_clean":"2023-10-06T00:22:19.073404+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:22:19.073404+0000","last_undegraded":"2023-10-06T00:22:19.073404+0000","last_fullsized":"2023-10-06T00:22:19.073404+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1904,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:22:15.970423+0000","last_change":"2023-10-06T00:22:15.970423+0000","last_active":"2023-10-06T00:22:15.970423+0000","last_peered":"2023-10-06T00:22:15.970423+0000","last_clean":"2023-10-06T00:22:15.970423+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:22:15.970423+0000","last_undegraded":"2023-10-06T00:22:15.970423+0000","last_fullsized":"2023-10-06T00:22:15.970423+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8618,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:18:25.215550+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:18:25.215550+0000","last_peered":"2023-10-06T00:18:25.215550+0000","last_clean":"2023-10-06T00:18:25.215550+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:18:25.215550+0000","last_undegraded":"2023-10-06T00:18:25.215550+0000","last_fullsized":"2023-10-06T00:18:25.215550+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 270s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11153,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:22:07.973445+0000","last_change":"2023-10-06T00:22:07.973445+0000","last_active":"2023-10-06T00:22:07.973445+0000","last_peered":"2023-10-06T00:22:07.973445+0000","last_clean":"2023-10-06T00:22:07.973445+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:22:07.973445+0000","last_undegraded":"2023-10-06T00:22:07.973445+0000","last_fullsized":"2023-10-06T00:22:07.973445+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2420,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:22:13.925572+0000","last_change":"2023-10-06T00:22:13.925572+0000","last_active":"2023-10-06T00:22:13.925572+0000","last_peered":"2023-10-06T00:22:13.925572+0000","last_clean":"2023-10-06T00:22:13.925572+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:22:13.925572+0000","last_undegraded":"2023-10-06T00:22:13.925572+0000","last_fullsized":"2023-10-06T00:22:13.925572+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11804,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:22:32.070367+0000","last_change":"2023-10-06T00:22:32.070367+0000","last_active":"2023-10-06T00:22:32.070367+0000","last_peered":"2023-10-06T00:22:32.070367+0000","last_clean":"2023-10-06T00:22:32.070367+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:22:32.070367+0000","last_undegraded":"2023-10-06T00:22:32.070367+0000","last_fullsized":"2023-10-06T00:22:32.070367+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11614,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:18:26.643373+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-06T00:18:26.643373+0000","last_peered":"2023-10-06T00:18:26.643373+0000","last_clean":"2023-10-06T00:18:26.643373+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-06T00:18:26.643373+0000","last_undegraded":"2023-10-06T00:18:26.643373+0000","last_fullsized":"2023-10-06T00:18:26.643373+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1784,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:18:27.028238+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-06T00:18:27.028238+0000","last_peered":"2023-10-06T00:18:27.028238+0000","last_clean":"2023-10-06T00:18:27.028238+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-06T00:18:27.028238+0000","last_undegraded":"2023-10-06T00:18:27.028238+0000","last_fullsized":"2023-10-06T00:18:27.028238+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1825,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:22:09.960041+0000","last_change":"2023-10-06T00:22:09.960041+0000","last_active":"2023-10-06T00:22:09.960041+0000","last_peered":"2023-10-06T00:22:09.960041+0000","last_clean":"2023-10-06T00:22:09.960041+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:22:09.960041+0000","last_undegraded":"2023-10-06T00:22:09.960041+0000","last_fullsized":"2023-10-06T00:22:09.960041+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740731,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.21199999999999999,"5min":0.20799999999999999,"15min":0.219},"min":{"1min":0.17000000000000001,"5min":0.13800000000000001,"15min":0.13800000000000001},"max":{"1min":0.26000000000000001,"5min":0.52100000000000002,"15min":1.153},"last":0.187},{"interface":"front","average":{"1min":0.246,"5min":0.24199999999999999,"15min":0.24199999999999999},"min":{"1min":0.19800000000000001,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.32200000000000001,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.23599999999999999}]},{"osd":1,"last update":"Fri Oct 6 00:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.28699999999999998,"5min":0.25800000000000001,"15min":0.252},"min":{"1min":0.21099999999999999,"5min":0.157,"15min":0.157},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.224},{"interface":"front","average":{"1min":0.27300000000000002,"5min":0.28000000000000003,"15min":0.27600000000000002},"min":{"1min":0.188,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.311,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.26800000000000002}]}]},{"osd":0,"up_from":8,"seq":34359740732,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:22:17 2023","interfaces":[{"interface":"back","average":{"1min":0.20200000000000001,"5min":0.20000000000000001,"15min":0.19700000000000001},"min":{"1min":0.14599999999999999,"5min":0.13900000000000001,"15min":0.128},"max":{"1min":0.377,"5min":0.377,"15min":0.377},"last":0.16800000000000001},{"interface":"front","average":{"1min":0.28499999999999998,"5min":0.26800000000000002,"15min":0.26600000000000001},"min":{"1min":0.218,"5min":0.17799999999999999,"15min":0.159},"max":{"1min":0.439,"5min":0.439,"15min":0.443},"last":0.26700000000000002}]},{"osd":2,"last update":"Fri Oct 6 00:22:17 2023","interfaces":[{"interface":"back","average":{"1min":0.252,"5min":0.23899999999999999,"15min":0.23599999999999999},"min":{"1min":0.18099999999999999,"5min":0.16600000000000001,"15min":0.16300000000000001},"max":{"1min":0.42199999999999999,"5min":0.438,"15min":0.438},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.313,"5min":0.29999999999999999,"15min":0.29799999999999999},"min":{"1min":0.24399999999999999,"5min":0.221,"15min":0.21299999999999999},"max":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.47499999999999998},"last":0.29599999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740727,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:21:38 2023","interfaces":[{"interface":"back","average":{"1min":0.20499999999999999,"5min":0.21099999999999999,"15min":0.21199999999999999},"min":{"1min":0.161,"5min":0.13100000000000001,"15min":0.13100000000000001},"max":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.497},"last":0.16300000000000001},{"interface":"front","average":{"1min":0.23599999999999999,"5min":0.24199999999999999,"15min":0.26000000000000001},"min":{"1min":0.17799999999999999,"5min":0.16300000000000001,"15min":0.154},"max":{"1min":0.40400000000000003,"5min":0.41199999999999998,"15min":0.51100000000000001},"last":0.19600000000000001}]},{"osd":2,"last update":"Fri Oct 6 00:21:38 2023","interfaces":[{"interface":"back","average":{"1min":0.24199999999999999,"5min":0.23799999999999999,"15min":0.23400000000000001},"min":{"1min":0.19700000000000001,"5min":0.18099999999999999,"15min":0.17100000000000001},"max":{"1min":0.434,"5min":0.44900000000000001,"15min":0.46999999999999997},"last":0.20899999999999999},{"interface":"front","average":{"1min":0.25700000000000001,"5min":0.24299999999999999,"15min":0.26200000000000001},"min":{"1min":0.20399999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.54700000000000004},"last":0.24399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:22:35.200 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:22:35.201 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:22:35.201 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:22:35.201 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:22:35.201 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:22:35.201 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:22:35.202 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:22:35.202 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:22:35.202 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:22:35.202 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.7 2023-10-06T00:22:35.537 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 2.7 on osd.1 to deep-scrub 2023-10-06T00:22:35.548 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.6 2023-10-06T00:22:35.845 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:35.843+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:35.907 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 2.6 on osd.1 to deep-scrub 2023-10-06T00:22:35.918 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.1 2023-10-06T00:22:36.097 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:36.095+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:36.277 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 3.1 on osd.0 to deep-scrub 2023-10-06T00:22:36.285 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.2 2023-10-06T00:22:36.649 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 3.2 on osd.1 to deep-scrub 2023-10-06T00:22:36.661 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.3 2023-10-06T00:22:36.870 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:36.869+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:37.019 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 2.3 on osd.1 to deep-scrub 2023-10-06T00:22:37.031 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.0 2023-10-06T00:22:37.138 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:37.137+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:37.419 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 3.0 on osd.1 to deep-scrub 2023-10-06T00:22:37.431 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.3 2023-10-06T00:22:37.787 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 3.3 on osd.0 to deep-scrub 2023-10-06T00:22:37.799 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.2 2023-10-06T00:22:37.866 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:37.865+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:38.097 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:38.095+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:38.167 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 2.2 on osd.0 to deep-scrub 2023-10-06T00:22:38.178 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.4 2023-10-06T00:22:38.559 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 2.4 on osd.1 to deep-scrub 2023-10-06T00:22:38.570 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:22:38.880 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:38.879+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:39.070 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:39.069+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:39.876 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:39.875+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:40.068 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:40.067+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:40.884 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:40.883+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:41.088 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:41.086+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:41.894 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:41.893+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:42.130 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:42.129+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:42.864 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:42.863+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:43.142 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:43.141+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:43.909 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:43.907+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:44.129 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:44.127+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:44.936 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:44.935+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:45.108 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:45.106+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:45.912 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:45.911+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:46.093 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:46.091+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:46.948 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:46.947+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:47.122 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:47.120+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:47.924 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:47.923+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:48.113 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:48.111+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:48.921 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:48.920+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:49.111 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:49.109+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:49.893 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:49.891+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:50.072 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:50.071+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:50.933 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:50.931+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:51.066 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:51.065+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:51.944 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:51.943+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:52.037 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:52.035+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:52.956 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:52.954+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:53.062 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:53.060+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:53.992 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:53.990+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:54.023 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:54.020+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:54.977 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:54.977+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:55.062 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:55.061+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:55.988 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:55.988+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:56.033 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:56.033+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:57.017 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:57.016+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:57.084 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:57.083+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:58.067 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:58.067+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:58.094 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:58.093+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:58.570 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:22:58.911 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:22:58.913 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:22:58.925 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":6052,"stamp":"2023-10-06T00:22:58.172954+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001817"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":2005,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:22:50.072170+0000","last_change":"2023-10-06T00:22:50.072170+0000","last_active":"2023-10-06T00:22:50.072170+0000","last_peered":"2023-10-06T00:22:50.072170+0000","last_clean":"2023-10-06T00:22:50.072170+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:22:50.072170+0000","last_undegraded":"2023-10-06T00:22:50.072170+0000","last_fullsized":"2023-10-06T00:22:50.072170+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1908,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:22:48.112639+0000","last_change":"2023-10-06T00:22:48.112639+0000","last_active":"2023-10-06T00:22:48.112639+0000","last_peered":"2023-10-06T00:22:48.112639+0000","last_clean":"2023-10-06T00:22:48.112639+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:22:48.112639+0000","last_undegraded":"2023-10-06T00:22:48.112639+0000","last_fullsized":"2023-10-06T00:22:48.112639+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8619,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:20:30.813100+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:20:30.813100+0000","last_peered":"2023-10-06T00:20:30.813100+0000","last_clean":"2023-10-06T00:20:30.813100+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:20:30.813100+0000","last_undegraded":"2023-10-06T00:20:30.813100+0000","last_fullsized":"2023-10-06T00:20:30.813100+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 395s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11157,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:22:40.068079+0000","last_change":"2023-10-06T00:22:40.068079+0000","last_active":"2023-10-06T00:22:40.068079+0000","last_peered":"2023-10-06T00:22:40.068079+0000","last_clean":"2023-10-06T00:22:40.068079+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:22:40.068079+0000","last_undegraded":"2023-10-06T00:22:40.068079+0000","last_fullsized":"2023-10-06T00:22:40.068079+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2424,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:22:45.107787+0000","last_change":"2023-10-06T00:22:45.107787+0000","last_active":"2023-10-06T00:22:45.107787+0000","last_peered":"2023-10-06T00:22:45.107787+0000","last_clean":"2023-10-06T00:22:45.107787+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:22:45.107787+0000","last_undegraded":"2023-10-06T00:22:45.107787+0000","last_fullsized":"2023-10-06T00:22:45.107787+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11808,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:22:51.066422+0000","last_change":"2023-10-06T00:22:51.066422+0000","last_active":"2023-10-06T00:22:51.066422+0000","last_peered":"2023-10-06T00:22:51.066422+0000","last_clean":"2023-10-06T00:22:51.066422+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:22:51.066422+0000","last_undegraded":"2023-10-06T00:22:51.066422+0000","last_fullsized":"2023-10-06T00:22:51.066422+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11615,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:20:32.244272+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-06T00:20:32.244272+0000","last_peered":"2023-10-06T00:20:32.244272+0000","last_clean":"2023-10-06T00:20:32.244272+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-06T00:20:32.244272+0000","last_undegraded":"2023-10-06T00:20:32.244272+0000","last_fullsized":"2023-10-06T00:20:32.244272+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1785,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:20:32.589628+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-06T00:20:32.589628+0000","last_peered":"2023-10-06T00:20:32.589628+0000","last_clean":"2023-10-06T00:20:32.589628+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-06T00:20:32.589628+0000","last_undegraded":"2023-10-06T00:20:32.589628+0000","last_fullsized":"2023-10-06T00:20:32.589628+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1829,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:22:44.128938+0000","last_change":"2023-10-06T00:22:44.128938+0000","last_active":"2023-10-06T00:22:44.128938+0000","last_peered":"2023-10-06T00:22:44.128938+0000","last_clean":"2023-10-06T00:22:44.128938+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:22:44.128938+0000","last_undegraded":"2023-10-06T00:22:44.128938+0000","last_fullsized":"2023-10-06T00:22:44.128938+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740736,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.21199999999999999,"5min":0.20799999999999999,"15min":0.219},"min":{"1min":0.17000000000000001,"5min":0.13800000000000001,"15min":0.13800000000000001},"max":{"1min":0.26000000000000001,"5min":0.52100000000000002,"15min":1.153},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.246,"5min":0.24199999999999999,"15min":0.24199999999999999},"min":{"1min":0.19800000000000001,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.32200000000000001,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.31900000000000001}]},{"osd":1,"last update":"Fri Oct 6 00:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.28699999999999998,"5min":0.25800000000000001,"15min":0.252},"min":{"1min":0.21099999999999999,"5min":0.157,"15min":0.157},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.27300000000000002,"5min":0.28000000000000003,"15min":0.27600000000000002},"min":{"1min":0.188,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.311,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.35899999999999999}]}]},{"osd":0,"up_from":8,"seq":34359740736,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:22:17 2023","interfaces":[{"interface":"back","average":{"1min":0.20200000000000001,"5min":0.20000000000000001,"15min":0.19700000000000001},"min":{"1min":0.14599999999999999,"5min":0.13900000000000001,"15min":0.128},"max":{"1min":0.377,"5min":0.377,"15min":0.377},"last":0.16600000000000001},{"interface":"front","average":{"1min":0.28499999999999998,"5min":0.26800000000000002,"15min":0.26600000000000001},"min":{"1min":0.218,"5min":0.17799999999999999,"15min":0.159},"max":{"1min":0.439,"5min":0.439,"15min":0.443},"last":0.23699999999999999}]},{"osd":2,"last update":"Fri Oct 6 00:22:17 2023","interfaces":[{"interface":"back","average":{"1min":0.252,"5min":0.23899999999999999,"15min":0.23599999999999999},"min":{"1min":0.18099999999999999,"5min":0.16600000000000001,"15min":0.16300000000000001},"max":{"1min":0.42199999999999999,"5min":0.438,"15min":0.438},"last":0.19600000000000001},{"interface":"front","average":{"1min":0.313,"5min":0.29999999999999999,"15min":0.29799999999999999},"min":{"1min":0.24399999999999999,"5min":0.221,"15min":0.21299999999999999},"max":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.47499999999999998},"last":0.26400000000000001}]}]},{"osd":1,"up_from":8,"seq":34359740731,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:22:43 2023","interfaces":[{"interface":"back","average":{"1min":0.20999999999999999,"5min":0.21099999999999999,"15min":0.21199999999999999},"min":{"1min":0.16,"5min":0.13100000000000001,"15min":0.13100000000000001},"max":{"1min":0.32500000000000001,"5min":0.39300000000000002,"15min":0.497},"last":0.17399999999999999},{"interface":"front","average":{"1min":0.22900000000000001,"5min":0.23999999999999999,"15min":0.25700000000000001},"min":{"1min":0.17899999999999999,"5min":0.16300000000000001,"15min":0.154},"max":{"1min":0.34399999999999997,"5min":0.41199999999999998,"15min":0.51100000000000001},"last":0.20200000000000001}]},{"osd":2,"last update":"Fri Oct 6 00:22:43 2023","interfaces":[{"interface":"back","average":{"1min":0.23799999999999999,"5min":0.23899999999999999,"15min":0.23499999999999999},"min":{"1min":0.193,"5min":0.18099999999999999,"15min":0.17100000000000001},"max":{"1min":0.36299999999999999,"5min":0.44900000000000001,"15min":0.46999999999999997},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.26000000000000001,"5min":0.249,"15min":0.26100000000000001},"min":{"1min":0.21099999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.39700000000000002,"5min":0.45500000000000002,"15min":0.54700000000000004},"last":0.253}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:22:58.927 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:22:58.927 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:22:58.927 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:22:58.927 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:22:58.928 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:22:58.928 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:22:58.928 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:22:58.928 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:22:58.928 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:22:58.929 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:22:59.101 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:22:59.100+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:22:59.134 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:22:59.133+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:00.065 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:00.065+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:00.164 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:00.163+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:01.060 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:01.059+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:01.160 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:01.159+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:02.078 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:02.077+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:02.153 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:02.152+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:03.098 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:03.097+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:03.179 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:03.179+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:04.136 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:04.135+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:04.204 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:04.203+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:05.174 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:05.173+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:05.175 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:05.175+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:06.192 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:06.191+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:06.218 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:06.218+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:07.192 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:07.191+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:07.223 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:07.223+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:08.159 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:08.159+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:08.197 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:08.196+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:09.168 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:09.168+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:09.216 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:09.216+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:10.181 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:10.180+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:10.252 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:10.252+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:11.224 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:11.223+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:11.236 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:11.235+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:12.175 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:12.175+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:12.229 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:12.228+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:13.163 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:13.162+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:13.231 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:13.231+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:14.152 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:14.151+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:14.184 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:14.183+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:15.180 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:15.178+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:15.187 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:15.186+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:16.182 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:16.181+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:16.207 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:16.206+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:17.172 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:17.170+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:17.215 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:17.214+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:18.185 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:18.184+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:18.246 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:18.244+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:18.929 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:23:19.228 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:19.226+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:19.274 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:19.273+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:19.294 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:23:19.296 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:23:19.307 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":6062,"stamp":"2023-10-06T00:23:18.176470+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002063"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":2005,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:22:50.072170+0000","last_change":"2023-10-06T00:22:50.072170+0000","last_active":"2023-10-06T00:22:50.072170+0000","last_peered":"2023-10-06T00:22:50.072170+0000","last_clean":"2023-10-06T00:22:50.072170+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:22:50.072170+0000","last_undegraded":"2023-10-06T00:22:50.072170+0000","last_fullsized":"2023-10-06T00:22:50.072170+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1911,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:23:07.191489+0000","last_change":"2023-10-06T00:23:07.191489+0000","last_active":"2023-10-06T00:23:07.191489+0000","last_peered":"2023-10-06T00:23:07.191489+0000","last_clean":"2023-10-06T00:23:07.191489+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:23:07.191489+0000","last_undegraded":"2023-10-06T00:23:07.191489+0000","last_fullsized":"2023-10-06T00:23:07.191489+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8619,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:20:30.813100+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:20:30.813100+0000","last_peered":"2023-10-06T00:20:30.813100+0000","last_clean":"2023-10-06T00:20:30.813100+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:20:30.813100+0000","last_undegraded":"2023-10-06T00:20:30.813100+0000","last_fullsized":"2023-10-06T00:20:30.813100+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 395s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11160,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:23:01.159372+0000","last_change":"2023-10-06T00:23:01.159372+0000","last_active":"2023-10-06T00:23:01.159372+0000","last_peered":"2023-10-06T00:23:01.159372+0000","last_clean":"2023-10-06T00:23:01.159372+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:23:01.159372+0000","last_undegraded":"2023-10-06T00:23:01.159372+0000","last_fullsized":"2023-10-06T00:23:01.159372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2427,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:23:05.173592+0000","last_change":"2023-10-06T00:23:05.173592+0000","last_active":"2023-10-06T00:23:05.173592+0000","last_peered":"2023-10-06T00:23:05.173592+0000","last_clean":"2023-10-06T00:23:05.173592+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:23:05.173592+0000","last_undegraded":"2023-10-06T00:23:05.173592+0000","last_fullsized":"2023-10-06T00:23:05.173592+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11808,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:22:51.066422+0000","last_change":"2023-10-06T00:22:51.066422+0000","last_active":"2023-10-06T00:22:51.066422+0000","last_peered":"2023-10-06T00:22:51.066422+0000","last_clean":"2023-10-06T00:22:51.066422+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:22:51.066422+0000","last_undegraded":"2023-10-06T00:22:51.066422+0000","last_fullsized":"2023-10-06T00:22:51.066422+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11615,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:20:32.244272+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-06T00:20:32.244272+0000","last_peered":"2023-10-06T00:20:32.244272+0000","last_clean":"2023-10-06T00:20:32.244272+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-06T00:20:32.244272+0000","last_undegraded":"2023-10-06T00:20:32.244272+0000","last_fullsized":"2023-10-06T00:20:32.244272+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1785,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:20:32.589628+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-06T00:20:32.589628+0000","last_peered":"2023-10-06T00:20:32.589628+0000","last_clean":"2023-10-06T00:20:32.589628+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-06T00:20:32.589628+0000","last_undegraded":"2023-10-06T00:20:32.589628+0000","last_fullsized":"2023-10-06T00:20:32.589628+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1832,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:23:04.203723+0000","last_change":"2023-10-06T00:23:04.203723+0000","last_active":"2023-10-06T00:23:04.203723+0000","last_peered":"2023-10-06T00:23:04.203723+0000","last_clean":"2023-10-06T00:23:04.203723+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:23:04.203723+0000","last_undegraded":"2023-10-06T00:23:04.203723+0000","last_fullsized":"2023-10-06T00:23:04.203723+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740740,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.21199999999999999,"5min":0.20799999999999999,"15min":0.219},"min":{"1min":0.17000000000000001,"5min":0.13800000000000001,"15min":0.13800000000000001},"max":{"1min":0.26000000000000001,"5min":0.52100000000000002,"15min":1.153},"last":0.224},{"interface":"front","average":{"1min":0.246,"5min":0.24199999999999999,"15min":0.24199999999999999},"min":{"1min":0.19800000000000001,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.32200000000000001,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.23899999999999999}]},{"osd":1,"last update":"Fri Oct 6 00:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.28699999999999998,"5min":0.25800000000000001,"15min":0.252},"min":{"1min":0.21099999999999999,"5min":0.157,"15min":0.157},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.249},{"interface":"front","average":{"1min":0.27300000000000002,"5min":0.28000000000000003,"15min":0.27600000000000002},"min":{"1min":0.188,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.311,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.20599999999999999}]}]},{"osd":0,"up_from":8,"seq":34359740740,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:22:17 2023","interfaces":[{"interface":"back","average":{"1min":0.20200000000000001,"5min":0.20000000000000001,"15min":0.19700000000000001},"min":{"1min":0.14599999999999999,"5min":0.13900000000000001,"15min":0.128},"max":{"1min":0.377,"5min":0.377,"15min":0.377},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.28499999999999998,"5min":0.26800000000000002,"15min":0.26600000000000001},"min":{"1min":0.218,"5min":0.17799999999999999,"15min":0.159},"max":{"1min":0.439,"5min":0.439,"15min":0.443},"last":0.38300000000000001}]},{"osd":2,"last update":"Fri Oct 6 00:22:17 2023","interfaces":[{"interface":"back","average":{"1min":0.252,"5min":0.23899999999999999,"15min":0.23599999999999999},"min":{"1min":0.18099999999999999,"5min":0.16600000000000001,"15min":0.16300000000000001},"max":{"1min":0.42199999999999999,"5min":0.438,"15min":0.438},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.313,"5min":0.29999999999999999,"15min":0.29799999999999999},"min":{"1min":0.24399999999999999,"5min":0.221,"15min":0.21299999999999999},"max":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.47499999999999998},"last":0.40999999999999998}]}]},{"osd":1,"up_from":8,"seq":34359740735,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:22:43 2023","interfaces":[{"interface":"back","average":{"1min":0.20999999999999999,"5min":0.21099999999999999,"15min":0.21199999999999999},"min":{"1min":0.16,"5min":0.13100000000000001,"15min":0.13100000000000001},"max":{"1min":0.32500000000000001,"5min":0.39300000000000002,"15min":0.497},"last":0.19},{"interface":"front","average":{"1min":0.22900000000000001,"5min":0.23999999999999999,"15min":0.25700000000000001},"min":{"1min":0.17899999999999999,"5min":0.16300000000000001,"15min":0.154},"max":{"1min":0.34399999999999997,"5min":0.41199999999999998,"15min":0.51100000000000001},"last":0.21199999999999999}]},{"osd":2,"last update":"Fri Oct 6 00:22:43 2023","interfaces":[{"interface":"back","average":{"1min":0.23799999999999999,"5min":0.23899999999999999,"15min":0.23499999999999999},"min":{"1min":0.193,"5min":0.18099999999999999,"15min":0.17100000000000001},"max":{"1min":0.36299999999999999,"5min":0.44900000000000001,"15min":0.46999999999999997},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.26000000000000001,"5min":0.249,"15min":0.26100000000000001},"min":{"1min":0.21099999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.39700000000000002,"5min":0.45500000000000002,"15min":0.54700000000000004},"last":0.26100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:23:19.308 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:23:19.309 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:23:19.309 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:23:19.309 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:23:19.309 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:23:19.310 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:23:19.310 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:23:19.310 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:23:19.310 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:23:19.310 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:23:20.183 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:20.182+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:20.242 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:20.241+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:21.200 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:21.198+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:21.257 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:21.256+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:22.193 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:22.191+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:22.273 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:22.271+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:23.231 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:23.230+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:23.246 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:23.245+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:24.251 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:24.249+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:24.261 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:24.260+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:25.273 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:25.271+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:25.301 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:25.300+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:26.287 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:26.286+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:26.288 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:26.287+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:27.290 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:27.288+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:27.337 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:27.336+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:28.325 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:28.324+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:28.345 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:28.344+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:29.303 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:29.302+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:29.340 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:29.339+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:30.297 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:30.295+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:30.386 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:30.384+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:31.280 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:31.279+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:31.424 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:31.423+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:32.318 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:32.316+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:32.454 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:32.452+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:33.298 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:33.296+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:33.446 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:33.445+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:34.315 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:34.314+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:34.462 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:34.460+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:35.288 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:35.286+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:35.452 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:35.451+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:36.319 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:36.318+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:36.414 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:36.412+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:37.330 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:37.328+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:37.415 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:37.414+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:38.341 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:38.339+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:38.411 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:38.410+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:39.312 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:23:39.341 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:39.340+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:39.437 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:39.435+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:39.674 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:23:39.675 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:23:39.687 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":6072,"stamp":"2023-10-06T00:23:38.179805+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001847"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":2008,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:23:18.245630+0000","last_change":"2023-10-06T00:23:18.245630+0000","last_active":"2023-10-06T00:23:18.245630+0000","last_peered":"2023-10-06T00:23:18.245630+0000","last_clean":"2023-10-06T00:23:18.245630+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:23:18.245630+0000","last_undegraded":"2023-10-06T00:23:18.245630+0000","last_fullsized":"2023-10-06T00:23:18.245630+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1911,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:23:07.191489+0000","last_change":"2023-10-06T00:23:07.191489+0000","last_active":"2023-10-06T00:23:07.191489+0000","last_peered":"2023-10-06T00:23:07.191489+0000","last_clean":"2023-10-06T00:23:07.191489+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:23:07.191489+0000","last_undegraded":"2023-10-06T00:23:07.191489+0000","last_fullsized":"2023-10-06T00:23:07.191489+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8619,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:20:30.813100+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:20:30.813100+0000","last_peered":"2023-10-06T00:20:30.813100+0000","last_clean":"2023-10-06T00:20:30.813100+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:20:30.813100+0000","last_undegraded":"2023-10-06T00:20:30.813100+0000","last_fullsized":"2023-10-06T00:20:30.813100+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 395s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11163,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:23:26.287202+0000","last_change":"2023-10-06T00:23:26.287202+0000","last_active":"2023-10-06T00:23:26.287202+0000","last_peered":"2023-10-06T00:23:26.287202+0000","last_clean":"2023-10-06T00:23:26.287202+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:23:26.287202+0000","last_undegraded":"2023-10-06T00:23:26.287202+0000","last_fullsized":"2023-10-06T00:23:26.287202+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2430,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:23:33.297518+0000","last_change":"2023-10-06T00:23:33.297518+0000","last_active":"2023-10-06T00:23:33.297518+0000","last_peered":"2023-10-06T00:23:33.297518+0000","last_clean":"2023-10-06T00:23:33.297518+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:23:33.297518+0000","last_undegraded":"2023-10-06T00:23:33.297518+0000","last_fullsized":"2023-10-06T00:23:33.297518+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11811,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:23:24.261094+0000","last_change":"2023-10-06T00:23:24.261094+0000","last_active":"2023-10-06T00:23:24.261094+0000","last_peered":"2023-10-06T00:23:24.261094+0000","last_clean":"2023-10-06T00:23:24.261094+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:23:24.261094+0000","last_undegraded":"2023-10-06T00:23:24.261094+0000","last_fullsized":"2023-10-06T00:23:24.261094+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11615,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:20:32.244272+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-06T00:20:32.244272+0000","last_peered":"2023-10-06T00:20:32.244272+0000","last_clean":"2023-10-06T00:20:32.244272+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-06T00:20:32.244272+0000","last_undegraded":"2023-10-06T00:20:32.244272+0000","last_fullsized":"2023-10-06T00:20:32.244272+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1785,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:20:32.589628+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-06T00:20:32.589628+0000","last_peered":"2023-10-06T00:20:32.589628+0000","last_clean":"2023-10-06T00:20:32.589628+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-06T00:20:32.589628+0000","last_undegraded":"2023-10-06T00:20:32.589628+0000","last_fullsized":"2023-10-06T00:20:32.589628+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1835,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:23:32.318098+0000","last_change":"2023-10-06T00:23:32.318098+0000","last_active":"2023-10-06T00:23:32.318098+0000","last_peered":"2023-10-06T00:23:32.318098+0000","last_clean":"2023-10-06T00:23:32.318098+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:23:32.318098+0000","last_undegraded":"2023-10-06T00:23:32.318098+0000","last_fullsized":"2023-10-06T00:23:32.318098+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740744,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:23:23 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.20799999999999999,"15min":0.22},"min":{"1min":0.158,"5min":0.13800000000000001,"15min":0.13800000000000001},"max":{"1min":0.375,"5min":0.52100000000000002,"15min":1.153},"last":0.17699999999999999},{"interface":"front","average":{"1min":0.25700000000000001,"5min":0.245,"15min":0.24299999999999999},"min":{"1min":0.185,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.41499999999999998,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.20599999999999999}]},{"osd":1,"last update":"Fri Oct 6 00:23:20 2023","interfaces":[{"interface":"back","average":{"1min":0.26600000000000001,"5min":0.25900000000000001,"15min":0.255},"min":{"1min":0.17699999999999999,"5min":0.157,"15min":0.157},"max":{"1min":0.40300000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.187},{"interface":"front","average":{"1min":0.28000000000000003,"5min":0.27800000000000002,"15min":0.27700000000000002},"min":{"1min":0.20599999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.47099999999999997,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.245}]}]},{"osd":0,"up_from":8,"seq":34359740744,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:23:21 2023","interfaces":[{"interface":"back","average":{"1min":0.19800000000000001,"5min":0.20000000000000001,"15min":0.19700000000000001},"min":{"1min":0.14199999999999999,"5min":0.13900000000000001,"15min":0.128},"max":{"1min":0.32700000000000001,"5min":0.377,"15min":0.377},"last":0.182},{"interface":"front","average":{"1min":0.26500000000000001,"5min":0.26800000000000002,"15min":0.26700000000000002},"min":{"1min":0.19400000000000001,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.38300000000000001,"5min":0.439,"15min":0.443},"last":0.27300000000000002}]},{"osd":2,"last update":"Fri Oct 6 00:23:21 2023","interfaces":[{"interface":"back","average":{"1min":0.23499999999999999,"5min":0.23999999999999999,"15min":0.23699999999999999},"min":{"1min":0.19600000000000001,"5min":0.16600000000000001,"15min":0.16300000000000001},"max":{"1min":0.35399999999999998,"5min":0.438,"15min":0.438},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.29799999999999999,"5min":0.30099999999999999,"15min":0.29899999999999999},"min":{"1min":0.26200000000000001,"5min":0.221,"15min":0.21299999999999999},"max":{"1min":0.40999999999999998,"5min":0.46300000000000002,"15min":0.47499999999999998},"last":0.29899999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740739,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:22:43 2023","interfaces":[{"interface":"back","average":{"1min":0.20999999999999999,"5min":0.21099999999999999,"15min":0.21199999999999999},"min":{"1min":0.16,"5min":0.13100000000000001,"15min":0.13100000000000001},"max":{"1min":0.32500000000000001,"5min":0.39300000000000002,"15min":0.497},"last":0.20000000000000001},{"interface":"front","average":{"1min":0.22900000000000001,"5min":0.23999999999999999,"15min":0.25700000000000001},"min":{"1min":0.17899999999999999,"5min":0.16300000000000001,"15min":0.154},"max":{"1min":0.34399999999999997,"5min":0.41199999999999998,"15min":0.51100000000000001},"last":0.189}]},{"osd":2,"last update":"Fri Oct 6 00:22:43 2023","interfaces":[{"interface":"back","average":{"1min":0.23799999999999999,"5min":0.23899999999999999,"15min":0.23499999999999999},"min":{"1min":0.193,"5min":0.18099999999999999,"15min":0.17100000000000001},"max":{"1min":0.36299999999999999,"5min":0.44900000000000001,"15min":0.46999999999999997},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.26000000000000001,"5min":0.249,"15min":0.26100000000000001},"min":{"1min":0.21099999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.39700000000000002,"5min":0.45500000000000002,"15min":0.54700000000000004},"last":0.23499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:23:39.688 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:23:39.689 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:23:39.689 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:23:39.689 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:23:39.689 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:23:39.690 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:23:39.690 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:23:39.690 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:23:39.690 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:23:39.690 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:23:40.312 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:40.310+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:40.420 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:40.418+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:41.292 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:41.290+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:41.441 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:41.439+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:42.262 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:42.260+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:42.439 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:42.437+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:43.281 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:43.280+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:43.395 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:43.394+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:44.262 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:44.260+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:44.407 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:44.405+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:45.293 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:45.291+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:45.361 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:45.360+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:46.249 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:46.247+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:46.402 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:46.400+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:47.231 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:47.230+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:47.389 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:47.387+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:48.227 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:48.226+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:48.352 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:48.350+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:49.254 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:49.252+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:49.383 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:49.381+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:50.280 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:50.279+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:50.343 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:50.341+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:51.316 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:51.314+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:51.333 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:51.332+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:52.277 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:52.276+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:52.337 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:52.336+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:53.245 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:53.245+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:53.338 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:53.337+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:54.220 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:54.219+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:54.330 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:54.329+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:55.196 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:55.196+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:55.364 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:55.363+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:56.174 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:56.173+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:56.414 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:56.414+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:57.151 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:57.151+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:57.366 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:57.365+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:58.149 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:58.149+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:58.370 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:58.369+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:59.181 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:23:59.181+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:59.335 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:23:59.335+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:23:59.691 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:24:00.067 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:24:00.069 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:24:00.080 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":6082,"stamp":"2023-10-06T00:23:58.183108+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002083"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":2014,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:23:51.315689+0000","last_change":"2023-10-06T00:23:51.315689+0000","last_active":"2023-10-06T00:23:51.315689+0000","last_peered":"2023-10-06T00:23:51.315689+0000","last_clean":"2023-10-06T00:23:51.315689+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:23:51.315689+0000","last_undegraded":"2023-10-06T00:23:51.315689+0000","last_fullsized":"2023-10-06T00:23:51.315689+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1917,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:23:50.279718+0000","last_change":"2023-10-06T00:23:50.279718+0000","last_active":"2023-10-06T00:23:50.279718+0000","last_peered":"2023-10-06T00:23:50.279718+0000","last_clean":"2023-10-06T00:23:50.279718+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:23:50.279718+0000","last_undegraded":"2023-10-06T00:23:50.279718+0000","last_fullsized":"2023-10-06T00:23:50.279718+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8619,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:20:30.813100+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:20:30.813100+0000","last_peered":"2023-10-06T00:20:30.813100+0000","last_clean":"2023-10-06T00:20:30.813100+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:20:30.813100+0000","last_undegraded":"2023-10-06T00:20:30.813100+0000","last_fullsized":"2023-10-06T00:20:30.813100+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 395s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11166,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:23:43.281148+0000","last_change":"2023-10-06T00:23:43.281148+0000","last_active":"2023-10-06T00:23:43.281148+0000","last_peered":"2023-10-06T00:23:43.281148+0000","last_clean":"2023-10-06T00:23:43.281148+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:23:43.281148+0000","last_undegraded":"2023-10-06T00:23:43.281148+0000","last_fullsized":"2023-10-06T00:23:43.281148+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2433,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:23:49.253856+0000","last_change":"2023-10-06T00:23:49.253856+0000","last_active":"2023-10-06T00:23:49.253856+0000","last_peered":"2023-10-06T00:23:49.253856+0000","last_clean":"2023-10-06T00:23:49.253856+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:23:49.253856+0000","last_undegraded":"2023-10-06T00:23:49.253856+0000","last_fullsized":"2023-10-06T00:23:49.253856+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11817,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:23:53.245294+0000","last_change":"2023-10-06T00:23:53.245294+0000","last_active":"2023-10-06T00:23:53.245294+0000","last_peered":"2023-10-06T00:23:53.245294+0000","last_clean":"2023-10-06T00:23:53.245294+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:23:53.245294+0000","last_undegraded":"2023-10-06T00:23:53.245294+0000","last_fullsized":"2023-10-06T00:23:53.245294+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11615,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:20:32.244272+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-06T00:20:32.244272+0000","last_peered":"2023-10-06T00:20:32.244272+0000","last_clean":"2023-10-06T00:20:32.244272+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-06T00:20:32.244272+0000","last_undegraded":"2023-10-06T00:20:32.244272+0000","last_fullsized":"2023-10-06T00:20:32.244272+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1785,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:20:32.589628+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-06T00:20:32.589628+0000","last_peered":"2023-10-06T00:20:32.589628+0000","last_clean":"2023-10-06T00:20:32.589628+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-06T00:20:32.589628+0000","last_undegraded":"2023-10-06T00:20:32.589628+0000","last_fullsized":"2023-10-06T00:20:32.589628+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1838,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:23:48.227222+0000","last_change":"2023-10-06T00:23:48.227222+0000","last_active":"2023-10-06T00:23:48.227222+0000","last_peered":"2023-10-06T00:23:48.227222+0000","last_clean":"2023-10-06T00:23:48.227222+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:23:48.227222+0000","last_undegraded":"2023-10-06T00:23:48.227222+0000","last_fullsized":"2023-10-06T00:23:48.227222+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740748,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:23:23 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.20799999999999999,"15min":0.22},"min":{"1min":0.158,"5min":0.13800000000000001,"15min":0.13800000000000001},"max":{"1min":0.375,"5min":0.52100000000000002,"15min":1.153},"last":0.19700000000000001},{"interface":"front","average":{"1min":0.25700000000000001,"5min":0.245,"15min":0.24299999999999999},"min":{"1min":0.185,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.41499999999999998,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.21299999999999999}]},{"osd":1,"last update":"Fri Oct 6 00:23:20 2023","interfaces":[{"interface":"back","average":{"1min":0.26600000000000001,"5min":0.25900000000000001,"15min":0.255},"min":{"1min":0.17699999999999999,"5min":0.157,"15min":0.157},"max":{"1min":0.40300000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.28000000000000003,"5min":0.27800000000000002,"15min":0.27700000000000002},"min":{"1min":0.20599999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.47099999999999997,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.253}]}]},{"osd":0,"up_from":8,"seq":34359740748,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:23:21 2023","interfaces":[{"interface":"back","average":{"1min":0.19800000000000001,"5min":0.20000000000000001,"15min":0.19700000000000001},"min":{"1min":0.14199999999999999,"5min":0.13900000000000001,"15min":0.128},"max":{"1min":0.32700000000000001,"5min":0.377,"15min":0.377},"last":0.17399999999999999},{"interface":"front","average":{"1min":0.26500000000000001,"5min":0.26800000000000002,"15min":0.26700000000000002},"min":{"1min":0.19400000000000001,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.38300000000000001,"5min":0.439,"15min":0.443},"last":0.22}]},{"osd":2,"last update":"Fri Oct 6 00:23:21 2023","interfaces":[{"interface":"back","average":{"1min":0.23499999999999999,"5min":0.23999999999999999,"15min":0.23699999999999999},"min":{"1min":0.19600000000000001,"5min":0.16600000000000001,"15min":0.16300000000000001},"max":{"1min":0.35399999999999998,"5min":0.438,"15min":0.438},"last":0.19500000000000001},{"interface":"front","average":{"1min":0.29799999999999999,"5min":0.30099999999999999,"15min":0.29899999999999999},"min":{"1min":0.26200000000000001,"5min":0.221,"15min":0.21299999999999999},"max":{"1min":0.40999999999999998,"5min":0.46300000000000002,"15min":0.47499999999999998},"last":0.253}]}]},{"osd":1,"up_from":8,"seq":34359740743,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.222,"5min":0.20899999999999999,"15min":0.21199999999999999},"min":{"1min":0.14899999999999999,"5min":0.13100000000000001,"15min":0.13100000000000001},"max":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.497},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.23799999999999999,"5min":0.23499999999999999,"15min":0.254},"min":{"1min":0.16700000000000001,"5min":0.16700000000000001,"15min":0.154},"max":{"1min":0.38900000000000001,"5min":0.41199999999999998,"15min":0.51100000000000001},"last":0.221}]},{"osd":2,"last update":"Fri Oct 6 00:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.25800000000000001,"5min":0.24299999999999999,"15min":0.23599999999999999},"min":{"1min":0.187,"5min":0.187,"15min":0.17100000000000001},"max":{"1min":0.40300000000000002,"5min":0.44900000000000001,"15min":0.46999999999999997},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.27700000000000002,"5min":0.25600000000000001,"15min":0.26200000000000001},"min":{"1min":0.19800000000000001,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.433,"5min":0.45500000000000002,"15min":0.54700000000000004},"last":0.27200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:24:00.082 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:24:00.082 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:24:00.082 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:24:00.083 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:24:00.083 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:24:00.083 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:24:00.083 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:24:00.083 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:24:00.083 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:24:00.084 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:24:00.182 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:00.181+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:00.317 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:00.316+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:01.183 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:01.183+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:01.307 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:01.306+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:02.210 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:02.209+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:02.295 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:02.295+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:03.242 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:03.241+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:03.339 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:03.338+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:04.257 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:04.257+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:04.369 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:04.369+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:05.218 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:05.218+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:05.367 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:05.366+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:06.242 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:06.241+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:06.389 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:06.388+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:07.273 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:07.272+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:07.419 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:07.418+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:08.285 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:08.285+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:08.409 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:08.408+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:09.287 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:09.286+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:09.454 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:09.454+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:10.246 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:10.245+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:10.437 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:10.437+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:11.204 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:11.203+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:11.395 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:11.395+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:12.214 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:12.214+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:12.439 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:12.438+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:13.169 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:13.169+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:13.438 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:13.436+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:14.130 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:14.128+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:14.450 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:14.448+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:15.121 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:15.119+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:15.445 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:15.443+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:16.110 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:16.108+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:16.465 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:16.464+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:17.118 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:17.116+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:17.421 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:17.417+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:18.091 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:18.090+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:18.385 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:18.384+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:19.076 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:19.074+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:19.423 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:19.422+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:20.084 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:24:20.105 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:20.104+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:20.397 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:20.395+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:20.425 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:24:20.428 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:24:20.439 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":6093,"stamp":"2023-10-06T00:24:20.186283+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001796"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":2020,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:24:15.120540+0000","last_change":"2023-10-06T00:24:15.120540+0000","last_active":"2023-10-06T00:24:15.120540+0000","last_peered":"2023-10-06T00:24:15.120540+0000","last_clean":"2023-10-06T00:24:15.120540+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:24:15.120540+0000","last_undegraded":"2023-10-06T00:24:15.120540+0000","last_fullsized":"2023-10-06T00:24:15.120540+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1923,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:24:14.129858+0000","last_change":"2023-10-06T00:24:14.129858+0000","last_active":"2023-10-06T00:24:14.129858+0000","last_peered":"2023-10-06T00:24:14.129858+0000","last_clean":"2023-10-06T00:24:14.129858+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:24:14.129858+0000","last_undegraded":"2023-10-06T00:24:14.129858+0000","last_fullsized":"2023-10-06T00:24:14.129858+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8619,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:20:30.813100+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:20:30.813100+0000","last_peered":"2023-10-06T00:20:30.813100+0000","last_clean":"2023-10-06T00:20:30.813100+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:20:30.813100+0000","last_undegraded":"2023-10-06T00:20:30.813100+0000","last_fullsized":"2023-10-06T00:20:30.813100+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 395s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11172,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:24:05.218086+0000","last_change":"2023-10-06T00:24:05.218086+0000","last_active":"2023-10-06T00:24:05.218086+0000","last_peered":"2023-10-06T00:24:05.218086+0000","last_clean":"2023-10-06T00:24:05.218086+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:24:05.218086+0000","last_undegraded":"2023-10-06T00:24:05.218086+0000","last_fullsized":"2023-10-06T00:24:05.218086+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2439,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:24:11.203569+0000","last_change":"2023-10-06T00:24:11.203569+0000","last_active":"2023-10-06T00:24:11.203569+0000","last_peered":"2023-10-06T00:24:11.203569+0000","last_clean":"2023-10-06T00:24:11.203569+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:24:11.203569+0000","last_undegraded":"2023-10-06T00:24:11.203569+0000","last_fullsized":"2023-10-06T00:24:11.203569+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11820,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:24:02.209584+0000","last_change":"2023-10-06T00:24:02.209584+0000","last_active":"2023-10-06T00:24:02.209584+0000","last_peered":"2023-10-06T00:24:02.209584+0000","last_clean":"2023-10-06T00:24:02.209584+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:24:02.209584+0000","last_undegraded":"2023-10-06T00:24:02.209584+0000","last_fullsized":"2023-10-06T00:24:02.209584+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11615,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:20:32.244272+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-06T00:20:32.244272+0000","last_peered":"2023-10-06T00:20:32.244272+0000","last_clean":"2023-10-06T00:20:32.244272+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-06T00:20:32.244272+0000","last_undegraded":"2023-10-06T00:20:32.244272+0000","last_fullsized":"2023-10-06T00:20:32.244272+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1785,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:20:32.589628+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-06T00:20:32.589628+0000","last_peered":"2023-10-06T00:20:32.589628+0000","last_clean":"2023-10-06T00:20:32.589628+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-06T00:20:32.589628+0000","last_undegraded":"2023-10-06T00:20:32.589628+0000","last_fullsized":"2023-10-06T00:20:32.589628+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1844,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:24:06.241770+0000","last_change":"2023-10-06T00:24:06.241770+0000","last_active":"2023-10-06T00:24:06.241770+0000","last_peered":"2023-10-06T00:24:06.241770+0000","last_clean":"2023-10-06T00:24:06.241770+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:24:06.241770+0000","last_undegraded":"2023-10-06T00:24:06.241770+0000","last_fullsized":"2023-10-06T00:24:06.241770+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740753,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:23:23 2023","interfaces":[{"interface":"back","average":{"1min":0.23400000000000001,"5min":0.20799999999999999,"15min":0.22},"min":{"1min":0.158,"5min":0.13800000000000001,"15min":0.13800000000000001},"max":{"1min":0.375,"5min":0.52100000000000002,"15min":1.153},"last":0.184},{"interface":"front","average":{"1min":0.25700000000000001,"5min":0.245,"15min":0.24299999999999999},"min":{"1min":0.185,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.41499999999999998,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.22}]},{"osd":1,"last update":"Fri Oct 6 00:23:20 2023","interfaces":[{"interface":"back","average":{"1min":0.26600000000000001,"5min":0.25900000000000001,"15min":0.255},"min":{"1min":0.17699999999999999,"5min":0.157,"15min":0.157},"max":{"1min":0.40300000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.23699999999999999},{"interface":"front","average":{"1min":0.28000000000000003,"5min":0.27800000000000002,"15min":0.27700000000000002},"min":{"1min":0.20599999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.47099999999999997,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.28199999999999997}]}]},{"osd":0,"up_from":8,"seq":34359740753,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:23:21 2023","interfaces":[{"interface":"back","average":{"1min":0.19800000000000001,"5min":0.20000000000000001,"15min":0.19700000000000001},"min":{"1min":0.14199999999999999,"5min":0.13900000000000001,"15min":0.128},"max":{"1min":0.32700000000000001,"5min":0.377,"15min":0.377},"last":0.14599999999999999},{"interface":"front","average":{"1min":0.26500000000000001,"5min":0.26800000000000002,"15min":0.26700000000000002},"min":{"1min":0.19400000000000001,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.38300000000000001,"5min":0.439,"15min":0.443},"last":0.20799999999999999}]},{"osd":2,"last update":"Fri Oct 6 00:23:21 2023","interfaces":[{"interface":"back","average":{"1min":0.23499999999999999,"5min":0.23999999999999999,"15min":0.23699999999999999},"min":{"1min":0.19600000000000001,"5min":0.16600000000000001,"15min":0.16300000000000001},"max":{"1min":0.35399999999999998,"5min":0.438,"15min":0.438},"last":0.186},{"interface":"front","average":{"1min":0.29799999999999999,"5min":0.30099999999999999,"15min":0.29899999999999999},"min":{"1min":0.26200000000000001,"5min":0.221,"15min":0.21299999999999999},"max":{"1min":0.40999999999999998,"5min":0.46300000000000002,"15min":0.47499999999999998},"last":0.24199999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740748,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.222,"5min":0.20899999999999999,"15min":0.21199999999999999},"min":{"1min":0.14899999999999999,"5min":0.13100000000000001,"15min":0.13100000000000001},"max":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.497},"last":0.221},{"interface":"front","average":{"1min":0.23799999999999999,"5min":0.23499999999999999,"15min":0.254},"min":{"1min":0.16700000000000001,"5min":0.16700000000000001,"15min":0.154},"max":{"1min":0.38900000000000001,"5min":0.41199999999999998,"15min":0.51100000000000001},"last":0.23699999999999999}]},{"osd":2,"last update":"Fri Oct 6 00:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.25800000000000001,"5min":0.24299999999999999,"15min":0.23599999999999999},"min":{"1min":0.187,"5min":0.187,"15min":0.17100000000000001},"max":{"1min":0.40300000000000002,"5min":0.44900000000000001,"15min":0.46999999999999997},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.27700000000000002,"5min":0.25600000000000001,"15min":0.26200000000000001},"min":{"1min":0.19800000000000001,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.433,"5min":0.45500000000000002,"15min":0.54700000000000004},"last":0.26500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:24:20.441 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:24:20.441 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:24:20.441 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:24:20.442 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:24:20.442 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:24:20.442 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:24:20.442 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:24:20.442 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:24:20.443 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:24:20.443 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:24:21.115 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:21.114+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:21.367 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:21.365+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:22.114 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:22.112+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:22.380 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:22.378+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:23.121 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:23.120+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:23.361 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:23.360+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:24.072 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:24.071+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:24.387 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:24.386+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:25.075 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:25.074+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:25.345 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:25.344+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:26.060 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:26.058+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:26.375 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:26.374+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:27.105 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:27.104+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:27.335 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:27.334+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:28.151 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:28.149+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:28.378 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:28.376+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:29.143 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:29.142+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:29.411 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:29.409+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:30.141 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:30.140+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:30.370 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:30.368+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:31.149 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:31.148+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:31.345 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:31.344+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:32.126 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:32.125+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:32.367 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:32.366+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:33.105 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:33.104+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:33.327 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:33.325+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:34.129 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:34.128+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:34.305 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:34.304+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:35.116 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:35.115+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:35.280 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:35.278+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:36.135 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:36.134+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:36.249 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:36.247+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:37.092 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:37.090+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:37.299 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:37.297+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:38.083 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:38.082+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:38.253 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:38.252+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:39.097 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:39.095+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:39.259 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:39.257+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:40.055 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:40.053+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:40.233 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:40.231+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:40.444 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:24:40.814 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:24:40.818 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:24:40.829 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":6103,"stamp":"2023-10-06T00:24:40.189346+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001857"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":2023,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:24:31.149247+0000","last_change":"2023-10-06T00:24:31.149247+0000","last_active":"2023-10-06T00:24:31.149247+0000","last_peered":"2023-10-06T00:24:31.149247+0000","last_clean":"2023-10-06T00:24:31.149247+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:24:31.149247+0000","last_undegraded":"2023-10-06T00:24:31.149247+0000","last_fullsized":"2023-10-06T00:24:31.149247+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1926,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:24:29.142832+0000","last_change":"2023-10-06T00:24:29.142832+0000","last_active":"2023-10-06T00:24:29.142832+0000","last_peered":"2023-10-06T00:24:29.142832+0000","last_clean":"2023-10-06T00:24:29.142832+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:24:29.142832+0000","last_undegraded":"2023-10-06T00:24:29.142832+0000","last_fullsized":"2023-10-06T00:24:29.142832+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8619,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:20:30.813100+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:20:30.813100+0000","last_peered":"2023-10-06T00:20:30.813100+0000","last_clean":"2023-10-06T00:20:30.813100+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:20:30.813100+0000","last_undegraded":"2023-10-06T00:20:30.813100+0000","last_fullsized":"2023-10-06T00:20:30.813100+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 395s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11178,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:24:36.135048+0000","last_change":"2023-10-06T00:24:36.135048+0000","last_active":"2023-10-06T00:24:36.135048+0000","last_peered":"2023-10-06T00:24:36.135048+0000","last_clean":"2023-10-06T00:24:36.135048+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:24:36.135048+0000","last_undegraded":"2023-10-06T00:24:36.135048+0000","last_fullsized":"2023-10-06T00:24:36.135048+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2445,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:24:38.083295+0000","last_change":"2023-10-06T00:24:38.083295+0000","last_active":"2023-10-06T00:24:38.083295+0000","last_peered":"2023-10-06T00:24:38.083295+0000","last_clean":"2023-10-06T00:24:38.083295+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:24:38.083295+0000","last_undegraded":"2023-10-06T00:24:38.083295+0000","last_fullsized":"2023-10-06T00:24:38.083295+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11826,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:24:35.116377+0000","last_change":"2023-10-06T00:24:35.116377+0000","last_active":"2023-10-06T00:24:35.116377+0000","last_peered":"2023-10-06T00:24:35.116377+0000","last_clean":"2023-10-06T00:24:35.116377+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:24:35.116377+0000","last_undegraded":"2023-10-06T00:24:35.116377+0000","last_fullsized":"2023-10-06T00:24:35.116377+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11615,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:20:32.244272+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-06T00:20:32.244272+0000","last_peered":"2023-10-06T00:20:32.244272+0000","last_clean":"2023-10-06T00:20:32.244272+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-06T00:20:32.244272+0000","last_undegraded":"2023-10-06T00:20:32.244272+0000","last_fullsized":"2023-10-06T00:20:32.244272+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1785,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:20:32.589628+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-06T00:20:32.589628+0000","last_peered":"2023-10-06T00:20:32.589628+0000","last_clean":"2023-10-06T00:20:32.589628+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-06T00:20:32.589628+0000","last_undegraded":"2023-10-06T00:20:32.589628+0000","last_fullsized":"2023-10-06T00:20:32.589628+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1850,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:24:37.091634+0000","last_change":"2023-10-06T00:24:37.091634+0000","last_active":"2023-10-06T00:24:37.091634+0000","last_peered":"2023-10-06T00:24:37.091634+0000","last_clean":"2023-10-06T00:24:37.091634+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:24:37.091634+0000","last_undegraded":"2023-10-06T00:24:37.091634+0000","last_fullsized":"2023-10-06T00:24:37.091634+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740757,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.217,"5min":0.21099999999999999,"15min":0.219},"min":{"1min":0.17699999999999999,"5min":0.13800000000000001,"15min":0.13800000000000001},"max":{"1min":0.32400000000000001,"5min":0.52100000000000002,"15min":1.153},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.245,"5min":0.249,"15min":0.24199999999999999},"min":{"1min":0.20599999999999999,"5min":0.17999999999999999,"15min":0.17299999999999999},"max":{"1min":0.35299999999999998,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.23699999999999999}]},{"osd":1,"last update":"Fri Oct 6 00:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.255,"5min":0.26200000000000001,"15min":0.255},"min":{"1min":0.187,"5min":0.17699999999999999,"15min":0.157},"max":{"1min":0.36299999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.28499999999999998,"5min":0.28000000000000003,"15min":0.27800000000000002},"min":{"1min":0.23000000000000001,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.40699999999999997,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.28199999999999997}]}]},{"osd":0,"up_from":8,"seq":34359740757,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.188,"5min":0.20000000000000001,"15min":0.19600000000000001},"min":{"1min":0.13900000000000001,"5min":0.13900000000000001,"15min":0.128},"max":{"1min":0.35399999999999998,"5min":0.377,"15min":0.377},"last":0.223},{"interface":"front","average":{"1min":0.26100000000000001,"5min":0.27000000000000002,"15min":0.26700000000000002},"min":{"1min":0.19700000000000001,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.42299999999999999,"5min":0.439,"15min":0.443},"last":0.32300000000000001}]},{"osd":2,"last update":"Fri Oct 6 00:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.22700000000000001,"5min":0.23899999999999999,"15min":0.23599999999999999},"min":{"1min":0.17699999999999999,"5min":0.16600000000000001,"15min":0.16300000000000001},"max":{"1min":0.38300000000000001,"5min":0.438,"15min":0.438},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.28899999999999998,"5min":0.30199999999999999,"15min":0.29799999999999999},"min":{"1min":0.23799999999999999,"5min":0.221,"15min":0.21299999999999999},"max":{"1min":0.44900000000000001,"5min":0.46300000000000002,"15min":0.47499999999999998},"last":0.35199999999999998}]}]},{"osd":1,"up_from":8,"seq":34359740752,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.222,"5min":0.20899999999999999,"15min":0.21199999999999999},"min":{"1min":0.14899999999999999,"5min":0.13100000000000001,"15min":0.13100000000000001},"max":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.497},"last":0.216},{"interface":"front","average":{"1min":0.23799999999999999,"5min":0.23499999999999999,"15min":0.254},"min":{"1min":0.16700000000000001,"5min":0.16700000000000001,"15min":0.154},"max":{"1min":0.38900000000000001,"5min":0.41199999999999998,"15min":0.51100000000000001},"last":0.224}]},{"osd":2,"last update":"Fri Oct 6 00:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.25800000000000001,"5min":0.24299999999999999,"15min":0.23599999999999999},"min":{"1min":0.187,"5min":0.187,"15min":0.17100000000000001},"max":{"1min":0.40300000000000002,"5min":0.44900000000000001,"15min":0.46999999999999997},"last":0.16800000000000001},{"interface":"front","average":{"1min":0.27700000000000002,"5min":0.25600000000000001,"15min":0.26200000000000001},"min":{"1min":0.19800000000000001,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.433,"5min":0.45500000000000002,"15min":0.54700000000000004},"last":0.19700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:24:40.831 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:24:40.831 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:24:40.831 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:24:40.831 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:24:40.832 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:24:40.832 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:24:40.832 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:24:40.832 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:24:40.833 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:24:40.833 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.7 2023-10-06T00:24:41.088 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:41.086+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:41.252 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:41.250+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:41.308 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 2.7 on osd.1 to deep-scrub 2023-10-06T00:24:41.320 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.6 2023-10-06T00:24:41.702 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 2.6 on osd.1 to deep-scrub 2023-10-06T00:24:41.714 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.1 2023-10-06T00:24:42.060 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:42.058+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:42.096 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 3.1 on osd.0 to deep-scrub 2023-10-06T00:24:42.108 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.2 2023-10-06T00:24:42.231 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:42.229+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:42.487 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 3.2 on osd.1 to deep-scrub 2023-10-06T00:24:42.498 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.3 2023-10-06T00:24:42.870 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 2.3 on osd.1 to deep-scrub 2023-10-06T00:24:42.881 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.0 2023-10-06T00:24:43.014 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:43.011+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:43.220 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 3.0 on osd.1 to deep-scrub 2023-10-06T00:24:43.232 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.3 2023-10-06T00:24:43.259 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:43.258+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:43.568 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 3.3 on osd.0 to deep-scrub 2023-10-06T00:24:43.579 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.2 2023-10-06T00:24:43.998 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 2.2 on osd.0 to deep-scrub 2023-10-06T00:24:44.008 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.4 2023-10-06T00:24:44.016 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:44.014+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:44.277 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:44.276+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:44.372 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 2.4 on osd.1 to deep-scrub 2023-10-06T00:24:44.385 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:24:44.998 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:44.996+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:45.245 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:45.243+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:45.955 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:45.953+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:46.247 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:46.246+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:46.960 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:46.959+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:47.297 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:47.295+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:47.969 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:47.968+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:48.249 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:48.247+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:48.971 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:48.969+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:49.242 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:49.240+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:49.930 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:49.928+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:50.271 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:50.270+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:50.944 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:50.942+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:51.308 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:51.308+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:51.938 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:51.938+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:52.287 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:52.287+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:52.961 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:52.961+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:53.333 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:53.333+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:53.957 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:53.957+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:54.293 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:54.293+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:54.963 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:54.962+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:55.264 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:55.264+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:55.916 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:55.915+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:56.247 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:56.247+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:56.885 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:56.885+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:57.212 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:57.212+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:57.846 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:57.845+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:58.219 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:58.218+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:58.877 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:58.877+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:59.241 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:24:59.240+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:24:59.915 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:24:59.914+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:00.227 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:00.227+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:00.954 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:00.953+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:01.240 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:01.239+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:01.966 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:01.966+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:02.286 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:02.286+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:02.952 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:02.951+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:03.258 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:03.258+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:03.999 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:03.998+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:04.287 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:04.287+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:04.387 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:25:04.750 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:25:04.753 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:25:04.764 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":6115,"stamp":"2023-10-06T00:25:04.193021+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001696"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":2030,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:24:49.929567+0000","last_change":"2023-10-06T00:24:49.929567+0000","last_active":"2023-10-06T00:24:49.929567+0000","last_peered":"2023-10-06T00:24:49.929567+0000","last_clean":"2023-10-06T00:24:49.929567+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:24:49.929567+0000","last_undegraded":"2023-10-06T00:24:49.929567+0000","last_fullsized":"2023-10-06T00:24:49.929567+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1933,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:24:48.970764+0000","last_change":"2023-10-06T00:24:48.970764+0000","last_active":"2023-10-06T00:24:48.970764+0000","last_peered":"2023-10-06T00:24:48.970764+0000","last_clean":"2023-10-06T00:24:48.970764+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:24:48.970764+0000","last_undegraded":"2023-10-06T00:24:48.970764+0000","last_fullsized":"2023-10-06T00:24:48.970764+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8620,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:22:36.270882+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:22:36.270882+0000","last_peered":"2023-10-06T00:22:36.270882+0000","last_clean":"2023-10-06T00:22:36.270882+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:22:36.270882+0000","last_undegraded":"2023-10-06T00:22:36.270882+0000","last_fullsized":"2023-10-06T00:22:36.270882+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 521s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11185,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:24:54.962588+0000","last_change":"2023-10-06T00:24:54.962588+0000","last_active":"2023-10-06T00:24:54.962588+0000","last_peered":"2023-10-06T00:24:54.962588+0000","last_clean":"2023-10-06T00:24:54.962588+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:24:54.962588+0000","last_undegraded":"2023-10-06T00:24:54.962588+0000","last_fullsized":"2023-10-06T00:24:54.962588+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2452,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:24:58.877119+0000","last_change":"2023-10-06T00:24:58.877119+0000","last_active":"2023-10-06T00:24:58.877119+0000","last_peered":"2023-10-06T00:24:58.877119+0000","last_clean":"2023-10-06T00:24:58.877119+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:24:58.877119+0000","last_undegraded":"2023-10-06T00:24:58.877119+0000","last_fullsized":"2023-10-06T00:24:58.877119+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11833,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:24:52.961075+0000","last_change":"2023-10-06T00:24:52.961075+0000","last_active":"2023-10-06T00:24:52.961075+0000","last_peered":"2023-10-06T00:24:52.961075+0000","last_clean":"2023-10-06T00:24:52.961075+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:24:52.961075+0000","last_undegraded":"2023-10-06T00:24:52.961075+0000","last_fullsized":"2023-10-06T00:24:52.961075+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11616,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:22:37.783338+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-06T00:22:37.783338+0000","last_peered":"2023-10-06T00:22:37.783338+0000","last_clean":"2023-10-06T00:22:37.783338+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-06T00:22:37.783338+0000","last_undegraded":"2023-10-06T00:22:37.783338+0000","last_fullsized":"2023-10-06T00:22:37.783338+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1786,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:22:38.163504+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-06T00:22:38.163504+0000","last_peered":"2023-10-06T00:22:38.163504+0000","last_clean":"2023-10-06T00:22:38.163504+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-06T00:22:38.163504+0000","last_undegraded":"2023-10-06T00:22:38.163504+0000","last_fullsized":"2023-10-06T00:22:38.163504+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1857,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:24:57.845733+0000","last_change":"2023-10-06T00:24:57.845733+0000","last_active":"2023-10-06T00:24:57.845733+0000","last_peered":"2023-10-06T00:24:57.845733+0000","last_clean":"2023-10-06T00:24:57.845733+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:24:57.845733+0000","last_undegraded":"2023-10-06T00:24:57.845733+0000","last_fullsized":"2023-10-06T00:24:57.845733+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740761,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.217,"5min":0.21099999999999999,"15min":0.219},"min":{"1min":0.17699999999999999,"5min":0.13800000000000001,"15min":0.13800000000000001},"max":{"1min":0.32400000000000001,"5min":0.52100000000000002,"15min":1.153},"last":0.16200000000000001},{"interface":"front","average":{"1min":0.245,"5min":0.249,"15min":0.24199999999999999},"min":{"1min":0.20599999999999999,"5min":0.17999999999999999,"15min":0.17299999999999999},"max":{"1min":0.35299999999999998,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.183}]},{"osd":1,"last update":"Fri Oct 6 00:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.255,"5min":0.26200000000000001,"15min":0.255},"min":{"1min":0.187,"5min":0.17699999999999999,"15min":0.157},"max":{"1min":0.36299999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.20000000000000001},{"interface":"front","average":{"1min":0.28499999999999998,"5min":0.28000000000000003,"15min":0.27800000000000002},"min":{"1min":0.23000000000000001,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.40699999999999997,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.22}]}]},{"osd":0,"up_from":8,"seq":34359740762,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.188,"5min":0.20000000000000001,"15min":0.19600000000000001},"min":{"1min":0.13900000000000001,"5min":0.13900000000000001,"15min":0.128},"max":{"1min":0.35399999999999998,"5min":0.377,"15min":0.377},"last":0.154},{"interface":"front","average":{"1min":0.26100000000000001,"5min":0.27000000000000002,"15min":0.26700000000000002},"min":{"1min":0.19700000000000001,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.42299999999999999,"5min":0.439,"15min":0.443},"last":0.27000000000000002}]},{"osd":2,"last update":"Fri Oct 6 00:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.22700000000000001,"5min":0.23899999999999999,"15min":0.23599999999999999},"min":{"1min":0.17699999999999999,"5min":0.16600000000000001,"15min":0.16300000000000001},"max":{"1min":0.38300000000000001,"5min":0.438,"15min":0.438},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.28899999999999998,"5min":0.30199999999999999,"15min":0.29799999999999999},"min":{"1min":0.23799999999999999,"5min":0.221,"15min":0.21299999999999999},"max":{"1min":0.44900000000000001,"5min":0.46300000000000002,"15min":0.47499999999999998},"last":0.30299999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740757,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.21099999999999999,"5min":0.20999999999999999,"15min":0.21199999999999999},"min":{"1min":0.16600000000000001,"5min":0.14499999999999999,"15min":0.13100000000000001},"max":{"1min":0.38200000000000001,"5min":0.44700000000000001,"15min":0.497},"last":0.184},{"interface":"front","average":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.252},"min":{"1min":0.16700000000000001,"5min":0.16700000000000001,"15min":0.154},"max":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.51100000000000001},"last":0.219}]},{"osd":2,"last update":"Fri Oct 6 00:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.24199999999999999,"5min":0.24399999999999999,"15min":0.23699999999999999},"min":{"1min":0.16800000000000001,"5min":0.16800000000000001,"15min":0.16800000000000001},"max":{"1min":0.437,"5min":0.437,"15min":0.46999999999999997},"last":0.23699999999999999},{"interface":"front","average":{"1min":0.253,"5min":0.25700000000000001,"15min":0.26100000000000001},"min":{"1min":0.19700000000000001,"5min":0.188,"15min":0.17199999999999999},"max":{"1min":0.42699999999999999,"5min":0.45500000000000002,"15min":0.54700000000000004},"last":0.253}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:25:04.766 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:25:04.766 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:25:04.766 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:25:04.767 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:25:04.767 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:25:04.767 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:25:04.767 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:25:04.767 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:25:04.768 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:25:04.768 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:25:04.971 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:04.971+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:05.333 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:05.333+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:05.991 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:05.990+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:06.355 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:06.355+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:06.985 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:06.985+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:07.315 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:07.315+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:08.022 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:08.022+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:08.282 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:08.282+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:08.999 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:08.999+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:09.291 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:09.291+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:09.952 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:09.952+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:10.319 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:10.319+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:10.905 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:10.904+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:11.323 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:11.323+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:11.858 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:11.856+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:12.295 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:12.294+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:12.905 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:12.904+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:13.323 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:13.322+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:13.907 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:13.905+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:14.280 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:14.278+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:14.864 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:14.862+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:15.291 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:15.289+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:15.851 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:15.849+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:16.320 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:16.318+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:16.881 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:16.880+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:17.350 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:17.348+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:17.882 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:17.880+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:18.379 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:18.377+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:18.833 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:18.832+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:19.388 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:19.386+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:19.821 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:19.820+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:20.362 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:20.360+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:20.780 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:20.778+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:21.386 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:21.384+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:21.795 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:21.794+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:22.348 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:22.346+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:22.747 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:22.746+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:23.320 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:23.318+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:23.770 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:23.768+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:24.341 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:24.339+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:24.769 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:25:24.782 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:24.780+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:25.099 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:25:25.102 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:25:25.113 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":6125,"stamp":"2023-10-06T00:25:24.196107+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001809"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":2033,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:25:06.985213+0000","last_change":"2023-10-06T00:25:06.985213+0000","last_active":"2023-10-06T00:25:06.985213+0000","last_peered":"2023-10-06T00:25:06.985213+0000","last_clean":"2023-10-06T00:25:06.985213+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:25:06.985213+0000","last_undegraded":"2023-10-06T00:25:06.985213+0000","last_fullsized":"2023-10-06T00:25:06.985213+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1936,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:25:03.998304+0000","last_change":"2023-10-06T00:25:03.998304+0000","last_active":"2023-10-06T00:25:03.998304+0000","last_peered":"2023-10-06T00:25:03.998304+0000","last_clean":"2023-10-06T00:25:03.998304+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:25:03.998304+0000","last_undegraded":"2023-10-06T00:25:03.998304+0000","last_fullsized":"2023-10-06T00:25:03.998304+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8620,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:22:36.270882+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:22:36.270882+0000","last_peered":"2023-10-06T00:22:36.270882+0000","last_clean":"2023-10-06T00:22:36.270882+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:22:36.270882+0000","last_undegraded":"2023-10-06T00:22:36.270882+0000","last_fullsized":"2023-10-06T00:22:36.270882+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 521s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11188,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:25:15.850616+0000","last_change":"2023-10-06T00:25:15.850616+0000","last_active":"2023-10-06T00:25:15.850616+0000","last_peered":"2023-10-06T00:25:15.850616+0000","last_clean":"2023-10-06T00:25:15.850616+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:25:15.850616+0000","last_undegraded":"2023-10-06T00:25:15.850616+0000","last_fullsized":"2023-10-06T00:25:15.850616+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2455,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:25:21.795227+0000","last_change":"2023-10-06T00:25:21.795227+0000","last_active":"2023-10-06T00:25:21.795227+0000","last_peered":"2023-10-06T00:25:21.795227+0000","last_clean":"2023-10-06T00:25:21.795227+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:25:21.795227+0000","last_undegraded":"2023-10-06T00:25:21.795227+0000","last_fullsized":"2023-10-06T00:25:21.795227+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11836,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:25:11.857814+0000","last_change":"2023-10-06T00:25:11.857814+0000","last_active":"2023-10-06T00:25:11.857814+0000","last_peered":"2023-10-06T00:25:11.857814+0000","last_clean":"2023-10-06T00:25:11.857814+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:25:11.857814+0000","last_undegraded":"2023-10-06T00:25:11.857814+0000","last_fullsized":"2023-10-06T00:25:11.857814+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11616,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:22:37.783338+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-06T00:22:37.783338+0000","last_peered":"2023-10-06T00:22:37.783338+0000","last_clean":"2023-10-06T00:22:37.783338+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-06T00:22:37.783338+0000","last_undegraded":"2023-10-06T00:22:37.783338+0000","last_fullsized":"2023-10-06T00:22:37.783338+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1786,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:22:38.163504+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-06T00:22:38.163504+0000","last_peered":"2023-10-06T00:22:38.163504+0000","last_clean":"2023-10-06T00:22:38.163504+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-06T00:22:38.163504+0000","last_undegraded":"2023-10-06T00:22:38.163504+0000","last_fullsized":"2023-10-06T00:22:38.163504+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1860,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:25:17.881461+0000","last_change":"2023-10-06T00:25:17.881461+0000","last_active":"2023-10-06T00:25:17.881461+0000","last_peered":"2023-10-06T00:25:17.881461+0000","last_clean":"2023-10-06T00:25:17.881461+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:25:17.881461+0000","last_undegraded":"2023-10-06T00:25:17.881461+0000","last_fullsized":"2023-10-06T00:25:17.881461+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740765,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.217,"5min":0.21099999999999999,"15min":0.219},"min":{"1min":0.17699999999999999,"5min":0.13800000000000001,"15min":0.13800000000000001},"max":{"1min":0.32400000000000001,"5min":0.52100000000000002,"15min":1.153},"last":0.21299999999999999},{"interface":"front","average":{"1min":0.245,"5min":0.249,"15min":0.24199999999999999},"min":{"1min":0.20599999999999999,"5min":0.17999999999999999,"15min":0.17299999999999999},"max":{"1min":0.35299999999999998,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.247}]},{"osd":1,"last update":"Fri Oct 6 00:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.255,"5min":0.26200000000000001,"15min":0.255},"min":{"1min":0.187,"5min":0.17699999999999999,"15min":0.157},"max":{"1min":0.36299999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.20000000000000001},{"interface":"front","average":{"1min":0.28499999999999998,"5min":0.28000000000000003,"15min":0.27800000000000002},"min":{"1min":0.23000000000000001,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.40699999999999997,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.28799999999999998}]}]},{"osd":0,"up_from":8,"seq":34359740766,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.188,"5min":0.20000000000000001,"15min":0.19600000000000001},"min":{"1min":0.13900000000000001,"5min":0.13900000000000001,"15min":0.128},"max":{"1min":0.35399999999999998,"5min":0.377,"15min":0.377},"last":0.16},{"interface":"front","average":{"1min":0.26100000000000001,"5min":0.27000000000000002,"15min":0.26700000000000002},"min":{"1min":0.19700000000000001,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.42299999999999999,"5min":0.439,"15min":0.443},"last":0.26600000000000001}]},{"osd":2,"last update":"Fri Oct 6 00:24:22 2023","interfaces":[{"interface":"back","average":{"1min":0.22700000000000001,"5min":0.23899999999999999,"15min":0.23599999999999999},"min":{"1min":0.17699999999999999,"5min":0.16600000000000001,"15min":0.16300000000000001},"max":{"1min":0.38300000000000001,"5min":0.438,"15min":0.438},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.28899999999999998,"5min":0.30199999999999999,"15min":0.29799999999999999},"min":{"1min":0.23799999999999999,"5min":0.221,"15min":0.21299999999999999},"max":{"1min":0.44900000000000001,"5min":0.46300000000000002,"15min":0.47499999999999998},"last":0.29699999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740761,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.21099999999999999,"5min":0.20999999999999999,"15min":0.21199999999999999},"min":{"1min":0.16600000000000001,"5min":0.14499999999999999,"15min":0.13100000000000001},"max":{"1min":0.38200000000000001,"5min":0.44700000000000001,"15min":0.497},"last":0.182},{"interface":"front","average":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.252},"min":{"1min":0.16700000000000001,"5min":0.16700000000000001,"15min":0.154},"max":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.51100000000000001},"last":0.20300000000000001}]},{"osd":2,"last update":"Fri Oct 6 00:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.24199999999999999,"5min":0.24399999999999999,"15min":0.23699999999999999},"min":{"1min":0.16800000000000001,"5min":0.16800000000000001,"15min":0.16800000000000001},"max":{"1min":0.437,"5min":0.437,"15min":0.46999999999999997},"last":0.22},{"interface":"front","average":{"1min":0.253,"5min":0.25700000000000001,"15min":0.26100000000000001},"min":{"1min":0.19700000000000001,"5min":0.188,"15min":0.17199999999999999},"max":{"1min":0.42699999999999999,"5min":0.45500000000000002,"15min":0.54700000000000004},"last":0.254}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:25:25.115 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:25:25.115 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:25:25.115 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:25:25.115 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:25:25.116 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:25:25.116 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:25:25.116 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:25:25.116 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:25:25.116 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:25:25.116 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:25:25.341 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:25.340+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:25.760 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:25.759+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:26.302 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:26.300+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:26.767 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:26.766+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:27.340 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:27.338+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:27.810 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:27.808+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:28.294 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:28.292+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:28.796 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:28.795+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:29.317 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:29.316+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:29.792 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:29.791+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:30.317 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:30.315+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:30.828 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:30.826+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:31.310 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:31.308+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:31.789 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:31.787+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:32.262 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:32.260+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:32.832 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:32.830+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:33.308 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:33.306+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:33.879 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:33.877+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:34.322 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:34.321+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:34.917 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:34.914+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:35.279 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:35.278+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:35.897 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:35.895+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:36.313 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:36.311+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:36.874 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:36.873+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:37.359 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:37.357+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:37.865 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:37.863+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:38.360 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:38.358+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:38.898 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:38.897+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:39.368 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:39.366+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:39.915 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:39.914+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:40.336 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:40.334+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:40.901 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:40.899+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:41.323 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:41.322+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:41.911 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:41.910+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:42.362 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:42.361+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:42.905 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:42.904+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:43.353 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:43.351+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:43.884 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:43.883+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:44.394 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:44.393+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:44.915 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:44.914+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:45.117 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:25:45.401 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:45.399+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:45.459 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:25:45.462 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:25:45.473 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":6135,"stamp":"2023-10-06T00:25:44.199305+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001986"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":2036,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:25:27.809902+0000","last_change":"2023-10-06T00:25:27.809902+0000","last_active":"2023-10-06T00:25:27.809902+0000","last_peered":"2023-10-06T00:25:27.809902+0000","last_clean":"2023-10-06T00:25:27.809902+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:25:27.809902+0000","last_undegraded":"2023-10-06T00:25:27.809902+0000","last_fullsized":"2023-10-06T00:25:27.809902+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1939,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:25:26.767004+0000","last_change":"2023-10-06T00:25:26.767004+0000","last_active":"2023-10-06T00:25:26.767004+0000","last_peered":"2023-10-06T00:25:26.767004+0000","last_clean":"2023-10-06T00:25:26.767004+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:25:26.767004+0000","last_undegraded":"2023-10-06T00:25:26.767004+0000","last_fullsized":"2023-10-06T00:25:26.767004+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8620,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:22:36.270882+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:22:36.270882+0000","last_peered":"2023-10-06T00:22:36.270882+0000","last_clean":"2023-10-06T00:22:36.270882+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:22:36.270882+0000","last_undegraded":"2023-10-06T00:22:36.270882+0000","last_fullsized":"2023-10-06T00:22:36.270882+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 521s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11191,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:25:41.911188+0000","last_change":"2023-10-06T00:25:41.911188+0000","last_active":"2023-10-06T00:25:41.911188+0000","last_peered":"2023-10-06T00:25:41.911188+0000","last_clean":"2023-10-06T00:25:41.911188+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:25:41.911188+0000","last_undegraded":"2023-10-06T00:25:41.911188+0000","last_fullsized":"2023-10-06T00:25:41.911188+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2455,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:25:21.795227+0000","last_change":"2023-10-06T00:25:21.795227+0000","last_active":"2023-10-06T00:25:21.795227+0000","last_peered":"2023-10-06T00:25:21.795227+0000","last_clean":"2023-10-06T00:25:21.795227+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:25:21.795227+0000","last_undegraded":"2023-10-06T00:25:21.795227+0000","last_fullsized":"2023-10-06T00:25:21.795227+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11839,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:25:35.896114+0000","last_change":"2023-10-06T00:25:35.896114+0000","last_active":"2023-10-06T00:25:35.896114+0000","last_peered":"2023-10-06T00:25:35.896114+0000","last_clean":"2023-10-06T00:25:35.896114+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:25:35.896114+0000","last_undegraded":"2023-10-06T00:25:35.896114+0000","last_fullsized":"2023-10-06T00:25:35.896114+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11616,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:22:37.783338+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-06T00:22:37.783338+0000","last_peered":"2023-10-06T00:22:37.783338+0000","last_clean":"2023-10-06T00:22:37.783338+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-06T00:22:37.783338+0000","last_undegraded":"2023-10-06T00:22:37.783338+0000","last_fullsized":"2023-10-06T00:22:37.783338+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1786,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:22:38.163504+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-06T00:22:38.163504+0000","last_peered":"2023-10-06T00:22:38.163504+0000","last_clean":"2023-10-06T00:22:38.163504+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-06T00:22:38.163504+0000","last_undegraded":"2023-10-06T00:22:38.163504+0000","last_fullsized":"2023-10-06T00:22:38.163504+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1863,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:25:42.904811+0000","last_change":"2023-10-06T00:25:42.904811+0000","last_active":"2023-10-06T00:25:42.904811+0000","last_peered":"2023-10-06T00:25:42.904811+0000","last_clean":"2023-10-06T00:25:42.904811+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:25:42.904811+0000","last_undegraded":"2023-10-06T00:25:42.904811+0000","last_fullsized":"2023-10-06T00:25:42.904811+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740769,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.25900000000000001,"5min":0.222,"15min":0.221},"min":{"1min":0.16200000000000001,"5min":0.152,"15min":0.13800000000000001},"max":{"1min":1.093,"5min":1.093,"15min":1.153},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.23200000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"min":{"1min":0.183,"5min":0.183,"15min":0.17299999999999999},"max":{"1min":0.30199999999999999,"5min":0.41499999999999998,"15min":1.1659999999999999},"last":0.34699999999999998}]},{"osd":1,"last update":"Fri Oct 6 00:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.28799999999999998,"5min":0.26900000000000002,"15min":0.25700000000000001},"min":{"1min":0.192,"5min":0.17699999999999999,"15min":0.157},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.27400000000000002,"5min":0.27200000000000002,"15min":0.27700000000000002},"min":{"1min":0.22,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.34000000000000002,"5min":0.47099999999999997,"15min":1.2110000000000001},"last":0.38}]}]},{"osd":0,"up_from":8,"seq":34359740770,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.20999999999999999,"5min":0.20399999999999999,"15min":0.19800000000000001},"min":{"1min":0.13600000000000001,"5min":0.13600000000000001,"15min":0.128},"max":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.21199999999999999},{"interface":"front","average":{"1min":0.28299999999999997,"5min":0.27400000000000002,"15min":0.26800000000000002},"min":{"1min":0.16500000000000001,"5min":0.16500000000000001,"15min":0.16500000000000001},"max":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.26600000000000001}]},{"osd":2,"last update":"Fri Oct 6 00:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.255,"5min":0.24399999999999999,"15min":0.23799999999999999},"min":{"1min":0.193,"5min":0.16600000000000001,"15min":0.16300000000000001},"max":{"1min":0.5,"5min":0.5,"15min":0.5},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.318,"5min":0.30599999999999999,"15min":0.29899999999999999},"min":{"1min":0.24199999999999999,"5min":0.221,"15min":0.21299999999999999},"max":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"last":0.29299999999999998}]}]},{"osd":1,"up_from":8,"seq":34359740765,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.21099999999999999,"5min":0.20999999999999999,"15min":0.21199999999999999},"min":{"1min":0.16600000000000001,"5min":0.14499999999999999,"15min":0.13100000000000001},"max":{"1min":0.38200000000000001,"5min":0.44700000000000001,"15min":0.497},"last":0.17699999999999999},{"interface":"front","average":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.252},"min":{"1min":0.16700000000000001,"5min":0.16700000000000001,"15min":0.154},"max":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.51100000000000001},"last":0.20300000000000001}]},{"osd":2,"last update":"Fri Oct 6 00:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.24199999999999999,"5min":0.24399999999999999,"15min":0.23699999999999999},"min":{"1min":0.16800000000000001,"5min":0.16800000000000001,"15min":0.16800000000000001},"max":{"1min":0.437,"5min":0.437,"15min":0.46999999999999997},"last":0.218},{"interface":"front","average":{"1min":0.253,"5min":0.25700000000000001,"15min":0.26100000000000001},"min":{"1min":0.19700000000000001,"5min":0.188,"15min":0.17199999999999999},"max":{"1min":0.42699999999999999,"5min":0.45500000000000002,"15min":0.54700000000000004},"last":0.25800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:25:45.475 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:25:45.475 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:25:45.475 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:25:45.476 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:25:45.476 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:25:45.476 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:25:45.476 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:25:45.476 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:25:45.477 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:25:45.477 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:25:45.903 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:45.902+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:46.353 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:46.352+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:46.919 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:46.917+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:47.370 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:47.368+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:47.966 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:47.965+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:48.413 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:48.412+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:48.971 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:48.969+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:49.396 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:49.395+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:50.006 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:50.005+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:50.358 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:50.357+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:50.989 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:50.989+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:51.392 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:51.391+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:52.007 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:52.007+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:52.369 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:52.368+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:52.993 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:52.992+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:53.347 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:53.347+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:54.007 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:54.006+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:54.343 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:54.342+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:55.043 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:55.043+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:55.387 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:55.387+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:56.091 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:56.091+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:56.355 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:56.355+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:57.088 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:57.087+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:57.353 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:57.352+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:58.077 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:58.077+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:58.376 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:58.375+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:59.124 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:25:59.124+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:25:59.420 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:25:59.420+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:00.124 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:00.123+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:00.393 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:00.393+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:01.083 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:01.082+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:01.411 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:01.410+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:02.048 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:02.047+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:02.439 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:02.439+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:03.058 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:03.058+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:03.433 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:03.433+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:04.044 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:04.043+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:04.420 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:04.419+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:05.092 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:05.092+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:05.463 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:05.462+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:05.477 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:26:05.855 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:26:05.857 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:26:05.869 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":6145,"stamp":"2023-10-06T00:26:04.202318+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001820"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":2042,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:26:02.047807+0000","last_change":"2023-10-06T00:26:02.047807+0000","last_active":"2023-10-06T00:26:02.047807+0000","last_peered":"2023-10-06T00:26:02.047807+0000","last_clean":"2023-10-06T00:26:02.047807+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:26:02.047807+0000","last_undegraded":"2023-10-06T00:26:02.047807+0000","last_fullsized":"2023-10-06T00:26:02.047807+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1945,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:26:00.123877+0000","last_change":"2023-10-06T00:26:00.123877+0000","last_active":"2023-10-06T00:26:00.123877+0000","last_peered":"2023-10-06T00:26:00.123877+0000","last_clean":"2023-10-06T00:26:00.123877+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:26:00.123877+0000","last_undegraded":"2023-10-06T00:26:00.123877+0000","last_fullsized":"2023-10-06T00:26:00.123877+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8620,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:22:36.270882+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:22:36.270882+0000","last_peered":"2023-10-06T00:22:36.270882+0000","last_clean":"2023-10-06T00:22:36.270882+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:22:36.270882+0000","last_undegraded":"2023-10-06T00:22:36.270882+0000","last_fullsized":"2023-10-06T00:22:36.270882+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 521s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11194,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:25:50.989161+0000","last_change":"2023-10-06T00:25:50.989161+0000","last_active":"2023-10-06T00:25:50.989161+0000","last_peered":"2023-10-06T00:25:50.989161+0000","last_clean":"2023-10-06T00:25:50.989161+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:25:50.989161+0000","last_undegraded":"2023-10-06T00:25:50.989161+0000","last_fullsized":"2023-10-06T00:25:50.989161+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2461,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:25:54.006485+0000","last_change":"2023-10-06T00:25:54.006485+0000","last_active":"2023-10-06T00:25:54.006485+0000","last_peered":"2023-10-06T00:25:54.006485+0000","last_clean":"2023-10-06T00:25:54.006485+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:25:54.006485+0000","last_undegraded":"2023-10-06T00:25:54.006485+0000","last_fullsized":"2023-10-06T00:25:54.006485+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11842,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:25:50.005783+0000","last_change":"2023-10-06T00:25:50.005783+0000","last_active":"2023-10-06T00:25:50.005783+0000","last_peered":"2023-10-06T00:25:50.005783+0000","last_clean":"2023-10-06T00:25:50.005783+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:25:50.005783+0000","last_undegraded":"2023-10-06T00:25:50.005783+0000","last_fullsized":"2023-10-06T00:25:50.005783+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11616,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:22:37.783338+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-06T00:22:37.783338+0000","last_peered":"2023-10-06T00:22:37.783338+0000","last_clean":"2023-10-06T00:22:37.783338+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-06T00:22:37.783338+0000","last_undegraded":"2023-10-06T00:22:37.783338+0000","last_fullsized":"2023-10-06T00:22:37.783338+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1786,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:22:38.163504+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-06T00:22:38.163504+0000","last_peered":"2023-10-06T00:22:38.163504+0000","last_clean":"2023-10-06T00:22:38.163504+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-06T00:22:38.163504+0000","last_undegraded":"2023-10-06T00:22:38.163504+0000","last_fullsized":"2023-10-06T00:22:38.163504+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1866,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:25:52.992455+0000","last_change":"2023-10-06T00:25:52.992455+0000","last_active":"2023-10-06T00:25:52.992455+0000","last_peered":"2023-10-06T00:25:52.992455+0000","last_clean":"2023-10-06T00:25:52.992455+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:25:52.992455+0000","last_undegraded":"2023-10-06T00:25:52.992455+0000","last_fullsized":"2023-10-06T00:25:52.992455+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740773,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.25900000000000001,"5min":0.222,"15min":0.221},"min":{"1min":0.16200000000000001,"5min":0.152,"15min":0.13800000000000001},"max":{"1min":1.093,"5min":1.093,"15min":1.153},"last":0.193},{"interface":"front","average":{"1min":0.23200000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"min":{"1min":0.183,"5min":0.183,"15min":0.17299999999999999},"max":{"1min":0.30199999999999999,"5min":0.41499999999999998,"15min":1.1659999999999999},"last":0.21099999999999999}]},{"osd":1,"last update":"Fri Oct 6 00:25:22 2023","interfaces":[{"interface":"back","average":{"1min":0.28799999999999998,"5min":0.26900000000000002,"15min":0.25700000000000001},"min":{"1min":0.192,"5min":0.17699999999999999,"15min":0.157},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.246},{"interface":"front","average":{"1min":0.27400000000000002,"5min":0.27200000000000002,"15min":0.27700000000000002},"min":{"1min":0.22,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.34000000000000002,"5min":0.47099999999999997,"15min":1.2110000000000001},"last":0.255}]}]},{"osd":0,"up_from":8,"seq":34359740774,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.20999999999999999,"5min":0.20399999999999999,"15min":0.19800000000000001},"min":{"1min":0.13600000000000001,"5min":0.13600000000000001,"15min":0.128},"max":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.28299999999999997,"5min":0.27400000000000002,"15min":0.26800000000000002},"min":{"1min":0.16500000000000001,"5min":0.16500000000000001,"15min":0.16500000000000001},"max":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.28699999999999998}]},{"osd":2,"last update":"Fri Oct 6 00:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.255,"5min":0.24399999999999999,"15min":0.23799999999999999},"min":{"1min":0.193,"5min":0.16600000000000001,"15min":0.16300000000000001},"max":{"1min":0.5,"5min":0.5,"15min":0.5},"last":0.246},{"interface":"front","average":{"1min":0.318,"5min":0.30599999999999999,"15min":0.29899999999999999},"min":{"1min":0.24199999999999999,"5min":0.221,"15min":0.21299999999999999},"max":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"last":0.315}]}]},{"osd":1,"up_from":8,"seq":34359740769,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.216,"5min":0.21199999999999999,"15min":0.21299999999999999},"min":{"1min":0.159,"5min":0.14899999999999999,"15min":0.13100000000000001},"max":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.497},"last":0.183},{"interface":"front","average":{"1min":0.22800000000000001,"5min":0.23200000000000001,"15min":0.249},"min":{"1min":0.17100000000000001,"5min":0.16700000000000001,"15min":0.154},"max":{"1min":0.432,"5min":0.432,"15min":0.51100000000000001},"last":0.20499999999999999}]},{"osd":2,"last update":"Fri Oct 6 00:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.23499999999999999,"5min":0.24299999999999999,"15min":0.23799999999999999},"min":{"1min":0.183,"5min":0.16800000000000001,"15min":0.16800000000000001},"max":{"1min":0.36499999999999999,"5min":0.437,"15min":0.46999999999999997},"last":0.222},{"interface":"front","average":{"1min":0.25600000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"min":{"1min":0.19800000000000001,"5min":0.19700000000000001,"15min":0.17199999999999999},"max":{"1min":0.375,"5min":0.45500000000000002,"15min":0.54700000000000004},"last":0.24199999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:26:05.871 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:26:05.871 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:26:05.871 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:26:05.871 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:26:05.872 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:26:05.872 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:26:05.872 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:26:05.872 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:26:05.872 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:26:05.872 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:26:06.126 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:06.125+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:06.485 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:06.485+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:07.122 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:07.121+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:07.481 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:07.480+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:08.108 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:08.108+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:08.518 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:08.518+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:09.158 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:09.157+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:09.543 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:09.542+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:10.205 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:10.203+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:10.518 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:10.517+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:11.165 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:11.163+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:11.492 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:11.490+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:12.117 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:12.115+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:12.494 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:12.493+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:13.079 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:13.077+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:13.513 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:13.511+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:14.075 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:14.073+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:14.544 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:14.542+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:15.102 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:15.100+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:15.509 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:15.507+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:16.121 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:16.120+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:16.546 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:16.545+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:17.079 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:17.077+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:17.497 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:17.496+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:18.034 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:18.033+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:18.532 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:18.531+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:19.008 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:19.006+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:19.538 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:19.536+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:20.024 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:20.023+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:20.530 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:20.528+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:21.050 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:21.049+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:21.493 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:21.491+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:22.045 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:22.044+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:22.459 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:22.457+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:23.030 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:23.028+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:23.435 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:23.433+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:24.058 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:24.056+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:24.412 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:24.410+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:25.084 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:25.082+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:25.440 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:25.439+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:25.873 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:26:26.067 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:26.065+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:26.333 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:26:26.336 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:26:26.348 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":6156,"stamp":"2023-10-06T00:26:26.205716+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001793"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":2045,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:26:19.007590+0000","last_change":"2023-10-06T00:26:19.007590+0000","last_active":"2023-10-06T00:26:19.007590+0000","last_peered":"2023-10-06T00:26:19.007590+0000","last_clean":"2023-10-06T00:26:19.007590+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:26:19.007590+0000","last_undegraded":"2023-10-06T00:26:19.007590+0000","last_fullsized":"2023-10-06T00:26:19.007590+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1948,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:26:18.034051+0000","last_change":"2023-10-06T00:26:18.034051+0000","last_active":"2023-10-06T00:26:18.034051+0000","last_peered":"2023-10-06T00:26:18.034051+0000","last_clean":"2023-10-06T00:26:18.034051+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:26:18.034051+0000","last_undegraded":"2023-10-06T00:26:18.034051+0000","last_fullsized":"2023-10-06T00:26:18.034051+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8620,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:22:36.270882+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:22:36.270882+0000","last_peered":"2023-10-06T00:22:36.270882+0000","last_clean":"2023-10-06T00:22:36.270882+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:22:36.270882+0000","last_undegraded":"2023-10-06T00:22:36.270882+0000","last_fullsized":"2023-10-06T00:22:36.270882+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 521s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11197,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:26:10.204346+0000","last_change":"2023-10-06T00:26:10.204346+0000","last_active":"2023-10-06T00:26:10.204346+0000","last_peered":"2023-10-06T00:26:10.204346+0000","last_clean":"2023-10-06T00:26:10.204346+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:26:10.204346+0000","last_undegraded":"2023-10-06T00:26:10.204346+0000","last_fullsized":"2023-10-06T00:26:10.204346+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2464,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:26:12.116451+0000","last_change":"2023-10-06T00:26:12.116451+0000","last_active":"2023-10-06T00:26:12.116451+0000","last_peered":"2023-10-06T00:26:12.116451+0000","last_clean":"2023-10-06T00:26:12.116451+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:26:12.116451+0000","last_undegraded":"2023-10-06T00:26:12.116451+0000","last_fullsized":"2023-10-06T00:26:12.116451+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11848,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:26:20.024180+0000","last_change":"2023-10-06T00:26:20.024180+0000","last_active":"2023-10-06T00:26:20.024180+0000","last_peered":"2023-10-06T00:26:20.024180+0000","last_clean":"2023-10-06T00:26:20.024180+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:26:20.024180+0000","last_undegraded":"2023-10-06T00:26:20.024180+0000","last_fullsized":"2023-10-06T00:26:20.024180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11616,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:22:37.783338+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-06T00:22:37.783338+0000","last_peered":"2023-10-06T00:22:37.783338+0000","last_clean":"2023-10-06T00:22:37.783338+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-06T00:22:37.783338+0000","last_undegraded":"2023-10-06T00:22:37.783338+0000","last_fullsized":"2023-10-06T00:22:37.783338+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1786,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:22:38.163504+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-06T00:22:38.163504+0000","last_peered":"2023-10-06T00:22:38.163504+0000","last_clean":"2023-10-06T00:22:38.163504+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-06T00:22:38.163504+0000","last_undegraded":"2023-10-06T00:22:38.163504+0000","last_fullsized":"2023-10-06T00:22:38.163504+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1869,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:26:11.164484+0000","last_change":"2023-10-06T00:26:11.164484+0000","last_active":"2023-10-06T00:26:11.164484+0000","last_peered":"2023-10-06T00:26:11.164484+0000","last_clean":"2023-10-06T00:26:11.164484+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:26:11.164484+0000","last_undegraded":"2023-10-06T00:26:11.164484+0000","last_fullsized":"2023-10-06T00:26:11.164484+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740778,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.25900000000000001,"5min":0.222,"15min":0.221},"min":{"1min":0.16200000000000001,"5min":0.152,"15min":0.13800000000000001},"max":{"1min":1.093,"5min":1.093,"15min":1.153},"last":0.17599999999999999},{"interface":"front","average":{"1min":0.23200000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"min":{"1min":0.183,"5min":0.183,"15min":0.17299999999999999},"max":{"1min":0.30199999999999999,"5min":0.41499999999999998,"15min":1.1659999999999999},"last":0.20300000000000001}]},{"osd":1,"last update":"Fri Oct 6 00:26:22 2023","interfaces":[{"interface":"back","average":{"1min":0.26900000000000002,"5min":0.27300000000000002,"15min":0.25900000000000001},"min":{"1min":0.19400000000000001,"5min":0.17699999999999999,"15min":0.157},"max":{"1min":0.45300000000000001,"5min":1.196,"15min":1.196},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.28299999999999997,"5min":0.27900000000000003,"15min":0.27800000000000002},"min":{"1min":0.19,"5min":0.188,"15min":0.17999999999999999},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":1.2110000000000001},"last":0.24299999999999999}]}]},{"osd":0,"up_from":8,"seq":34359740778,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.20999999999999999,"5min":0.20399999999999999,"15min":0.19800000000000001},"min":{"1min":0.13600000000000001,"5min":0.13600000000000001,"15min":0.128},"max":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.152},{"interface":"front","average":{"1min":0.28299999999999997,"5min":0.27400000000000002,"15min":0.26800000000000002},"min":{"1min":0.16500000000000001,"5min":0.16500000000000001,"15min":0.16500000000000001},"max":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.216}]},{"osd":2,"last update":"Fri Oct 6 00:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.255,"5min":0.24399999999999999,"15min":0.23799999999999999},"min":{"1min":0.193,"5min":0.16600000000000001,"15min":0.16300000000000001},"max":{"1min":0.5,"5min":0.5,"15min":0.5},"last":0.182},{"interface":"front","average":{"1min":0.318,"5min":0.30599999999999999,"15min":0.29899999999999999},"min":{"1min":0.24199999999999999,"5min":0.221,"15min":0.21299999999999999},"max":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"last":0.23999999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740773,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.216,"5min":0.21199999999999999,"15min":0.21299999999999999},"min":{"1min":0.159,"5min":0.14899999999999999,"15min":0.13100000000000001},"max":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.497},"last":0.22},{"interface":"front","average":{"1min":0.22800000000000001,"5min":0.23200000000000001,"15min":0.249},"min":{"1min":0.17100000000000001,"5min":0.16700000000000001,"15min":0.154},"max":{"1min":0.432,"5min":0.432,"15min":0.51100000000000001},"last":0.192}]},{"osd":2,"last update":"Fri Oct 6 00:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.23499999999999999,"5min":0.24299999999999999,"15min":0.23799999999999999},"min":{"1min":0.183,"5min":0.16800000000000001,"15min":0.16800000000000001},"max":{"1min":0.36499999999999999,"5min":0.437,"15min":0.46999999999999997},"last":0.249},{"interface":"front","average":{"1min":0.25600000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"min":{"1min":0.19800000000000001,"5min":0.19700000000000001,"15min":0.17199999999999999},"max":{"1min":0.375,"5min":0.45500000000000002,"15min":0.54700000000000004},"last":0.20300000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:26:26.349 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:26:26.350 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:26:26.350 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:26:26.350 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:26:26.350 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:26:26.350 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:26:26.351 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:26:26.351 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:26:26.351 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:26:26.351 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:26:26.483 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:26.481+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:27.113 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:27.112+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:27.467 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:27.465+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:28.159 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:28.157+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:28.457 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:28.455+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:29.142 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:29.140+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:29.428 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:29.426+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:30.161 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:30.159+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:30.416 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:30.414+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:31.204 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:31.202+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:31.410 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:31.409+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:32.184 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:32.183+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:32.440 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:32.439+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:33.193 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:33.192+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:33.431 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:33.429+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:34.186 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:34.184+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:34.441 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:34.439+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:35.222 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:35.220+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:35.408 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:35.407+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:36.249 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:36.247+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:36.450 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:36.449+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:37.202 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:37.200+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:37.418 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:37.417+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:38.243 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:38.241+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:38.448 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:38.447+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:39.215 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:39.213+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:39.462 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:39.460+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:40.264 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:40.263+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:40.493 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:40.492+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:41.235 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:41.233+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:41.463 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:41.461+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:42.214 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:42.213+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:42.508 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:42.506+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:43.263 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:43.262+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:43.503 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:43.502+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:44.265 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:44.263+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:44.494 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:44.492+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:45.299 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:45.298+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:45.446 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:45.444+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:46.295 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:46.293+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:46.352 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:26:46.429 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:46.427+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:46.713 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:26:46.716 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:26:46.730 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":6166,"stamp":"2023-10-06T00:26:46.208534+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001701"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":2048,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:26:34.185700+0000","last_change":"2023-10-06T00:26:34.185700+0000","last_active":"2023-10-06T00:26:34.185700+0000","last_peered":"2023-10-06T00:26:34.185700+0000","last_clean":"2023-10-06T00:26:34.185700+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:26:34.185700+0000","last_undegraded":"2023-10-06T00:26:34.185700+0000","last_fullsized":"2023-10-06T00:26:34.185700+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1951,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:26:33.193092+0000","last_change":"2023-10-06T00:26:33.193092+0000","last_active":"2023-10-06T00:26:33.193092+0000","last_peered":"2023-10-06T00:26:33.193092+0000","last_clean":"2023-10-06T00:26:33.193092+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:26:33.193092+0000","last_undegraded":"2023-10-06T00:26:33.193092+0000","last_fullsized":"2023-10-06T00:26:33.193092+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8620,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:22:36.270882+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:22:36.270882+0000","last_peered":"2023-10-06T00:22:36.270882+0000","last_clean":"2023-10-06T00:22:36.270882+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:22:36.270882+0000","last_undegraded":"2023-10-06T00:22:36.270882+0000","last_fullsized":"2023-10-06T00:22:36.270882+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 521s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11203,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:26:37.201260+0000","last_change":"2023-10-06T00:26:37.201260+0000","last_active":"2023-10-06T00:26:37.201260+0000","last_peered":"2023-10-06T00:26:37.201260+0000","last_clean":"2023-10-06T00:26:37.201260+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:26:37.201260+0000","last_undegraded":"2023-10-06T00:26:37.201260+0000","last_fullsized":"2023-10-06T00:26:37.201260+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2467,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:26:30.160293+0000","last_change":"2023-10-06T00:26:30.160293+0000","last_active":"2023-10-06T00:26:30.160293+0000","last_peered":"2023-10-06T00:26:30.160293+0000","last_clean":"2023-10-06T00:26:30.160293+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:26:30.160293+0000","last_undegraded":"2023-10-06T00:26:30.160293+0000","last_fullsized":"2023-10-06T00:26:30.160293+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11851,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:26:36.249455+0000","last_change":"2023-10-06T00:26:36.249455+0000","last_active":"2023-10-06T00:26:36.249455+0000","last_peered":"2023-10-06T00:26:36.249455+0000","last_clean":"2023-10-06T00:26:36.249455+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:26:36.249455+0000","last_undegraded":"2023-10-06T00:26:36.249455+0000","last_fullsized":"2023-10-06T00:26:36.249455+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11616,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:22:37.783338+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-06T00:22:37.783338+0000","last_peered":"2023-10-06T00:22:37.783338+0000","last_clean":"2023-10-06T00:22:37.783338+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-06T00:22:37.783338+0000","last_undegraded":"2023-10-06T00:22:37.783338+0000","last_fullsized":"2023-10-06T00:22:37.783338+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1786,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:22:38.163504+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-06T00:22:38.163504+0000","last_peered":"2023-10-06T00:22:38.163504+0000","last_clean":"2023-10-06T00:22:38.163504+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-06T00:22:38.163504+0000","last_undegraded":"2023-10-06T00:22:38.163504+0000","last_fullsized":"2023-10-06T00:22:38.163504+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1875,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:26:42.214344+0000","last_change":"2023-10-06T00:26:42.214344+0000","last_active":"2023-10-06T00:26:42.214344+0000","last_peered":"2023-10-06T00:26:42.214344+0000","last_clean":"2023-10-06T00:26:42.214344+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:26:42.214344+0000","last_undegraded":"2023-10-06T00:26:42.214344+0000","last_fullsized":"2023-10-06T00:26:42.214344+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740782,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:26:29 2023","interfaces":[{"interface":"back","average":{"1min":0.215,"5min":0.22700000000000001,"15min":0.222},"min":{"1min":0.159,"5min":0.158,"15min":0.13800000000000001},"max":{"1min":0.40200000000000002,"5min":1.093,"15min":1.153},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.247,"5min":0.245,"15min":0.24199999999999999},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.17299999999999999},"max":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":1.1659999999999999},"last":0.32700000000000001}]},{"osd":1,"last update":"Fri Oct 6 00:26:22 2023","interfaces":[{"interface":"back","average":{"1min":0.26900000000000002,"5min":0.27300000000000002,"15min":0.25900000000000001},"min":{"1min":0.19400000000000001,"5min":0.17699999999999999,"15min":0.157},"max":{"1min":0.45300000000000001,"5min":1.196,"15min":1.196},"last":0.22},{"interface":"front","average":{"1min":0.28299999999999997,"5min":0.27900000000000003,"15min":0.27800000000000002},"min":{"1min":0.19,"5min":0.188,"15min":0.17999999999999999},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":1.2110000000000001},"last":0.28999999999999998}]}]},{"osd":0,"up_from":8,"seq":34359740782,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:26:26 2023","interfaces":[{"interface":"back","average":{"1min":0.187,"5min":0.19700000000000001,"15min":0.19800000000000001},"min":{"1min":0.13700000000000001,"5min":0.13600000000000001,"15min":0.128},"max":{"1min":0.41999999999999998,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.16},{"interface":"front","average":{"1min":0.26200000000000001,"5min":0.27100000000000002,"15min":0.26900000000000002},"min":{"1min":0.19900000000000001,"5min":0.16500000000000001,"15min":0.16500000000000001},"max":{"1min":0.441,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.23699999999999999}]},{"osd":2,"last update":"Fri Oct 6 00:26:26 2023","interfaces":[{"interface":"back","average":{"1min":0.23000000000000001,"5min":0.23899999999999999,"15min":0.23799999999999999},"min":{"1min":0.16800000000000001,"5min":0.16800000000000001,"15min":0.16300000000000001},"max":{"1min":0.40500000000000003,"5min":0.5,"15min":0.5},"last":0.20300000000000001},{"interface":"front","average":{"1min":0.28899999999999998,"5min":0.30099999999999999,"15min":0.29999999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.21299999999999999},"max":{"1min":0.46300000000000002,"5min":0.53800000000000003,"15min":0.53800000000000003},"last":0.27000000000000002}]}]},{"osd":1,"up_from":8,"seq":34359740777,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.216,"5min":0.21199999999999999,"15min":0.21299999999999999},"min":{"1min":0.159,"5min":0.14899999999999999,"15min":0.13100000000000001},"max":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.497},"last":0.14699999999999999},{"interface":"front","average":{"1min":0.22800000000000001,"5min":0.23200000000000001,"15min":0.249},"min":{"1min":0.17100000000000001,"5min":0.16700000000000001,"15min":0.154},"max":{"1min":0.432,"5min":0.432,"15min":0.51100000000000001},"last":0.20300000000000001}]},{"osd":2,"last update":"Fri Oct 6 00:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.23499999999999999,"5min":0.24299999999999999,"15min":0.23799999999999999},"min":{"1min":0.183,"5min":0.16800000000000001,"15min":0.16800000000000001},"max":{"1min":0.36499999999999999,"5min":0.437,"15min":0.46999999999999997},"last":0.183},{"interface":"front","average":{"1min":0.25600000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"min":{"1min":0.19800000000000001,"5min":0.19700000000000001,"15min":0.17199999999999999},"max":{"1min":0.375,"5min":0.45500000000000002,"15min":0.54700000000000004},"last":0.217}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:26:46.731 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:26:46.732 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:26:46.732 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:26:46.732 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:26:46.732 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:26:46.732 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:26:46.733 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:26:46.733 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:26:46.733 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:26:46.733 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.7 2023-10-06T00:26:47.111 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 2.7 on osd.1 to deep-scrub 2023-10-06T00:26:47.123 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.6 2023-10-06T00:26:47.324 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:47.323+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:47.416 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:47.415+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:47.505 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 2.6 on osd.1 to deep-scrub 2023-10-06T00:26:47.517 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.1 2023-10-06T00:26:47.876 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 3.1 on osd.0 to deep-scrub 2023-10-06T00:26:47.888 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.2 2023-10-06T00:26:48.262 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 3.2 on osd.1 to deep-scrub 2023-10-06T00:26:48.273 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.3 2023-10-06T00:26:48.318 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:48.318+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:48.434 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:48.434+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:48.648 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 2.3 on osd.1 to deep-scrub 2023-10-06T00:26:48.660 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.0 2023-10-06T00:26:48.987 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 3.0 on osd.1 to deep-scrub 2023-10-06T00:26:48.999 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.3 2023-10-06T00:26:49.285 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:49.284+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:49.388 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 3.3 on osd.0 to deep-scrub 2023-10-06T00:26:49.400 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.2 2023-10-06T00:26:49.467 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:49.466+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:49.780 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 2.2 on osd.0 to deep-scrub 2023-10-06T00:26:49.792 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.4 2023-10-06T00:26:50.127 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 2.4 on osd.1 to deep-scrub 2023-10-06T00:26:50.136 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:26:50.306 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:50.306+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:50.469 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:50.469+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:51.288 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:51.288+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:51.497 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:51.497+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:52.243 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:52.243+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:52.520 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:52.520+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:53.225 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:53.225+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:53.545 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:53.544+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:54.209 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:54.208+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:54.532 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:54.531+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:55.191 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:55.191+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:55.511 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:55.510+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:56.151 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:56.150+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:56.481 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:56.480+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:57.133 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:57.132+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:57.514 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:57.513+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:58.090 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:58.089+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:58.558 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:58.557+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:59.132 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:26:59.131+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:26:59.604 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:26:59.603+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:00.107 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:00.106+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:00.579 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:00.579+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:01.088 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:01.088+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:01.629 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:01.628+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:02.091 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:02.091+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:02.663 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:02.662+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:03.050 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:03.049+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:03.673 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:03.673+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:04.038 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:04.037+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:04.717 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:04.714+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:05.067 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:05.067+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:05.673 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:05.672+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:06.085 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:06.085+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:06.637 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:06.636+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:07.041 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:07.041+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:07.633 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:07.633+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:08.194 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:08.082+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:08.587 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:08.585+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:09.047 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:09.046+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:09.600 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:09.598+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:10.036 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:10.035+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:10.137 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:27:10.514 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:27:10.516 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:27:10.527 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":6178,"stamp":"2023-10-06T00:27:10.212297+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001738"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":2052,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:26:52.243117+0000","last_change":"2023-10-06T00:26:52.243117+0000","last_active":"2023-10-06T00:26:52.243117+0000","last_peered":"2023-10-06T00:26:52.243117+0000","last_clean":"2023-10-06T00:26:52.243117+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:26:52.243117+0000","last_undegraded":"2023-10-06T00:26:52.243117+0000","last_fullsized":"2023-10-06T00:26:52.243117+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1958,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:27:06.085327+0000","last_change":"2023-10-06T00:27:06.085327+0000","last_active":"2023-10-06T00:27:06.085327+0000","last_peered":"2023-10-06T00:27:06.085327+0000","last_clean":"2023-10-06T00:27:06.085327+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:27:06.085327+0000","last_undegraded":"2023-10-06T00:27:06.085327+0000","last_fullsized":"2023-10-06T00:27:06.085327+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8621,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:24:42.092671+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:24:42.092671+0000","last_peered":"2023-10-06T00:24:42.092671+0000","last_clean":"2023-10-06T00:24:42.092671+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:24:42.092671+0000","last_undegraded":"2023-10-06T00:24:42.092671+0000","last_fullsized":"2023-10-06T00:24:42.092671+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 647s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11207,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:26:58.089730+0000","last_change":"2023-10-06T00:26:58.089730+0000","last_active":"2023-10-06T00:26:58.089730+0000","last_peered":"2023-10-06T00:26:58.089730+0000","last_clean":"2023-10-06T00:26:58.089730+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:26:58.089730+0000","last_undegraded":"2023-10-06T00:26:58.089730+0000","last_fullsized":"2023-10-06T00:26:58.089730+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2474,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:27:03.049576+0000","last_change":"2023-10-06T00:27:03.049576+0000","last_active":"2023-10-06T00:27:03.049576+0000","last_peered":"2023-10-06T00:27:03.049576+0000","last_clean":"2023-10-06T00:27:03.049576+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:27:03.049576+0000","last_undegraded":"2023-10-06T00:27:03.049576+0000","last_fullsized":"2023-10-06T00:27:03.049576+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11855,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:26:53.224981+0000","last_change":"2023-10-06T00:26:53.224981+0000","last_active":"2023-10-06T00:26:53.224981+0000","last_peered":"2023-10-06T00:26:53.224981+0000","last_clean":"2023-10-06T00:26:53.224981+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:26:53.224981+0000","last_undegraded":"2023-10-06T00:26:53.224981+0000","last_fullsized":"2023-10-06T00:26:53.224981+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11617,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:24:43.565128+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-06T00:24:43.565128+0000","last_peered":"2023-10-06T00:24:43.565128+0000","last_clean":"2023-10-06T00:24:43.565128+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-06T00:24:43.565128+0000","last_undegraded":"2023-10-06T00:24:43.565128+0000","last_fullsized":"2023-10-06T00:24:43.565128+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1787,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:24:43.994138+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-06T00:24:43.994138+0000","last_peered":"2023-10-06T00:24:43.994138+0000","last_clean":"2023-10-06T00:24:43.994138+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-06T00:24:43.994138+0000","last_undegraded":"2023-10-06T00:24:43.994138+0000","last_fullsized":"2023-10-06T00:24:43.994138+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1879,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:26:59.131846+0000","last_change":"2023-10-06T00:26:59.131846+0000","last_active":"2023-10-06T00:26:59.131846+0000","last_peered":"2023-10-06T00:26:59.131846+0000","last_clean":"2023-10-06T00:26:59.131846+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:26:59.131846+0000","last_undegraded":"2023-10-06T00:26:59.131846+0000","last_fullsized":"2023-10-06T00:26:59.131846+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740787,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:26:29 2023","interfaces":[{"interface":"back","average":{"1min":0.215,"5min":0.22700000000000001,"15min":0.222},"min":{"1min":0.159,"5min":0.158,"15min":0.13800000000000001},"max":{"1min":0.40200000000000002,"5min":1.093,"15min":1.153},"last":0.20399999999999999},{"interface":"front","average":{"1min":0.247,"5min":0.245,"15min":0.24199999999999999},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.17299999999999999},"max":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":1.1659999999999999},"last":0.21299999999999999}]},{"osd":1,"last update":"Fri Oct 6 00:26:22 2023","interfaces":[{"interface":"back","average":{"1min":0.26900000000000002,"5min":0.27300000000000002,"15min":0.25900000000000001},"min":{"1min":0.19400000000000001,"5min":0.17699999999999999,"15min":0.157},"max":{"1min":0.45300000000000001,"5min":1.196,"15min":1.196},"last":0.192},{"interface":"front","average":{"1min":0.28299999999999997,"5min":0.27900000000000003,"15min":0.27800000000000002},"min":{"1min":0.19,"5min":0.188,"15min":0.17999999999999999},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":1.2110000000000001},"last":0.251}]}]},{"osd":0,"up_from":8,"seq":34359740787,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:26:26 2023","interfaces":[{"interface":"back","average":{"1min":0.187,"5min":0.19700000000000001,"15min":0.19800000000000001},"min":{"1min":0.13700000000000001,"5min":0.13600000000000001,"15min":0.128},"max":{"1min":0.41999999999999998,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.20499999999999999},{"interface":"front","average":{"1min":0.26200000000000001,"5min":0.27100000000000002,"15min":0.26900000000000002},"min":{"1min":0.19900000000000001,"5min":0.16500000000000001,"15min":0.16500000000000001},"max":{"1min":0.441,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.28699999999999998}]},{"osd":2,"last update":"Fri Oct 6 00:26:26 2023","interfaces":[{"interface":"back","average":{"1min":0.23000000000000001,"5min":0.23899999999999999,"15min":0.23799999999999999},"min":{"1min":0.16800000000000001,"5min":0.16800000000000001,"15min":0.16300000000000001},"max":{"1min":0.40500000000000003,"5min":0.5,"15min":0.5},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.28899999999999998,"5min":0.30099999999999999,"15min":0.29999999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.21299999999999999},"max":{"1min":0.46300000000000002,"5min":0.53800000000000003,"15min":0.53800000000000003},"last":0.373}]}]},{"osd":1,"up_from":8,"seq":34359740782,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:26:53 2023","interfaces":[{"interface":"back","average":{"1min":0.191,"5min":0.20999999999999999,"15min":0.21299999999999999},"min":{"1min":0.14000000000000001,"5min":0.14000000000000001,"15min":0.13100000000000001},"max":{"1min":0.29599999999999999,"5min":0.45800000000000002,"15min":0.497},"last":0.20699999999999999},{"interface":"front","average":{"1min":0.214,"5min":0.22800000000000001,"15min":0.246},"min":{"1min":0.16700000000000001,"5min":0.16700000000000001,"15min":0.154},"max":{"1min":0.27800000000000002,"5min":0.432,"15min":0.51100000000000001},"last":0.22700000000000001}]},{"osd":2,"last update":"Fri Oct 6 00:26:53 2023","interfaces":[{"interface":"back","average":{"1min":0.22800000000000001,"5min":0.23999999999999999,"15min":0.23799999999999999},"min":{"1min":0.17899999999999999,"5min":0.16800000000000001,"15min":0.16800000000000001},"max":{"1min":0.30499999999999999,"5min":0.437,"15min":0.46999999999999997},"last":0.245},{"interface":"front","average":{"1min":0.24099999999999999,"5min":0.25700000000000001,"15min":0.25800000000000001},"min":{"1min":0.19900000000000001,"5min":0.19700000000000001,"15min":0.17199999999999999},"max":{"1min":0.316,"5min":0.433,"15min":0.54700000000000004},"last":0.25600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:27:10.529 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:27:10.529 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:27:10.529 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:27:10.530 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:27:10.530 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:27:10.530 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:27:10.530 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:27:10.530 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:27:10.531 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:27:10.531 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:27:10.609 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:10.607+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:11.032 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:11.030+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:11.651 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:11.649+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:12.014 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:12.013+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:12.689 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:12.687+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:12.976 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:12.975+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:13.728 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:13.727+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:13.977 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:13.976+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:14.757 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:14.755+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:14.985 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:14.984+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:15.728 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:15.726+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:15.987 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:15.986+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:16.736 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:16.734+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:17.006 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:17.004+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:17.728 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:17.726+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:18.040 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:18.038+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:18.774 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:18.772+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:19.052 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:19.050+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:19.729 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:19.727+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:20.051 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:20.049+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:20.683 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:20.681+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:21.101 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:21.099+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:21.649 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:21.647+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:22.097 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:22.095+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:22.630 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:22.628+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:23.076 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:23.074+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:23.652 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:23.650+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:24.072 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:24.070+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:24.678 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:24.676+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:25.114 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:25.113+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:25.659 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:25.658+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:26.142 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:26.140+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:26.707 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:26.705+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:27.183 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:27.182+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:27.738 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:27.736+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:28.224 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:28.223+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:28.724 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:28.722+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:29.259 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:29.258+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:29.722 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:29.721+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:30.258 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:30.256+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:30.531 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:27:30.724 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:30.723+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:30.901 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:27:30.904 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:27:30.915 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":6188,"stamp":"2023-10-06T00:27:30.215614+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002092"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":2055,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:27:11.031712+0000","last_change":"2023-10-06T00:27:11.031712+0000","last_active":"2023-10-06T00:27:11.031712+0000","last_peered":"2023-10-06T00:27:11.031712+0000","last_clean":"2023-10-06T00:27:11.031712+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:27:11.031712+0000","last_undegraded":"2023-10-06T00:27:11.031712+0000","last_fullsized":"2023-10-06T00:27:11.031712+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1958,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:27:06.085327+0000","last_change":"2023-10-06T00:27:06.085327+0000","last_active":"2023-10-06T00:27:06.085327+0000","last_peered":"2023-10-06T00:27:06.085327+0000","last_clean":"2023-10-06T00:27:06.085327+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:27:06.085327+0000","last_undegraded":"2023-10-06T00:27:06.085327+0000","last_fullsized":"2023-10-06T00:27:06.085327+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8621,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:24:42.092671+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:24:42.092671+0000","last_peered":"2023-10-06T00:24:42.092671+0000","last_clean":"2023-10-06T00:24:42.092671+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:24:42.092671+0000","last_undegraded":"2023-10-06T00:24:42.092671+0000","last_fullsized":"2023-10-06T00:24:42.092671+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 647s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11210,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:27:21.100758+0000","last_change":"2023-10-06T00:27:21.100758+0000","last_active":"2023-10-06T00:27:21.100758+0000","last_peered":"2023-10-06T00:27:21.100758+0000","last_clean":"2023-10-06T00:27:21.100758+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:27:21.100758+0000","last_undegraded":"2023-10-06T00:27:21.100758+0000","last_fullsized":"2023-10-06T00:27:21.100758+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2477,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:27:28.223787+0000","last_change":"2023-10-06T00:27:28.223787+0000","last_active":"2023-10-06T00:27:28.223787+0000","last_peered":"2023-10-06T00:27:28.223787+0000","last_clean":"2023-10-06T00:27:28.223787+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:27:28.223787+0000","last_undegraded":"2023-10-06T00:27:28.223787+0000","last_fullsized":"2023-10-06T00:27:28.223787+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11858,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:27:12.976233+0000","last_change":"2023-10-06T00:27:12.976233+0000","last_active":"2023-10-06T00:27:12.976233+0000","last_peered":"2023-10-06T00:27:12.976233+0000","last_clean":"2023-10-06T00:27:12.976233+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:27:12.976233+0000","last_undegraded":"2023-10-06T00:27:12.976233+0000","last_fullsized":"2023-10-06T00:27:12.976233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11617,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:24:43.565128+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-06T00:24:43.565128+0000","last_peered":"2023-10-06T00:24:43.565128+0000","last_clean":"2023-10-06T00:24:43.565128+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-06T00:24:43.565128+0000","last_undegraded":"2023-10-06T00:24:43.565128+0000","last_fullsized":"2023-10-06T00:24:43.565128+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1787,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:24:43.994138+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-06T00:24:43.994138+0000","last_peered":"2023-10-06T00:24:43.994138+0000","last_clean":"2023-10-06T00:24:43.994138+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-06T00:24:43.994138+0000","last_undegraded":"2023-10-06T00:24:43.994138+0000","last_fullsized":"2023-10-06T00:24:43.994138+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1882,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:27:25.114100+0000","last_change":"2023-10-06T00:27:25.114100+0000","last_active":"2023-10-06T00:27:25.114100+0000","last_peered":"2023-10-06T00:27:25.114100+0000","last_clean":"2023-10-06T00:27:25.114100+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:27:25.114100+0000","last_undegraded":"2023-10-06T00:27:25.114100+0000","last_fullsized":"2023-10-06T00:27:25.114100+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740791,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:26:29 2023","interfaces":[{"interface":"back","average":{"1min":0.215,"5min":0.22700000000000001,"15min":0.222},"min":{"1min":0.159,"5min":0.158,"15min":0.13800000000000001},"max":{"1min":0.40200000000000002,"5min":1.093,"15min":1.153},"last":0.19600000000000001},{"interface":"front","average":{"1min":0.247,"5min":0.245,"15min":0.24199999999999999},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.17299999999999999},"max":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":1.1659999999999999},"last":0.221}]},{"osd":1,"last update":"Fri Oct 6 00:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.251,"5min":0.26500000000000001,"15min":0.25800000000000001},"min":{"1min":0.186,"5min":0.17699999999999999,"15min":0.157},"max":{"1min":0.38200000000000001,"5min":1.196,"15min":1.196},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.27200000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"min":{"1min":0.19400000000000001,"5min":0.19,"15min":0.17999999999999999},"max":{"1min":0.45800000000000002,"5min":0.48199999999999998,"15min":1.2110000000000001},"last":0.247}]}]},{"osd":0,"up_from":8,"seq":34359740791,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.19700000000000001,"5min":0.19600000000000001,"15min":0.19900000000000001},"min":{"1min":0.16,"5min":0.13600000000000001,"15min":0.13600000000000001},"max":{"1min":0.35099999999999998,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.26400000000000001,"5min":0.26700000000000002,"15min":0.26900000000000002},"min":{"1min":0.17399999999999999,"5min":0.16500000000000001,"15min":0.16500000000000001},"max":{"1min":0.44800000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.217}]},{"osd":2,"last update":"Fri Oct 6 00:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.249,"5min":0.23899999999999999,"15min":0.23999999999999999},"min":{"1min":0.20100000000000001,"5min":0.16800000000000001,"15min":0.16600000000000001},"max":{"1min":0.41699999999999998,"5min":0.5,"15min":0.5},"last":0.20399999999999999},{"interface":"front","average":{"1min":0.30299999999999999,"5min":0.29899999999999999,"15min":0.29999999999999999},"min":{"1min":0.22800000000000001,"5min":0.217,"15min":0.21299999999999999},"max":{"1min":0.47699999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"last":0.27800000000000002}]}]},{"osd":1,"up_from":8,"seq":34359740786,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:26:53 2023","interfaces":[{"interface":"back","average":{"1min":0.191,"5min":0.20999999999999999,"15min":0.21299999999999999},"min":{"1min":0.14000000000000001,"5min":0.14000000000000001,"15min":0.13100000000000001},"max":{"1min":0.29599999999999999,"5min":0.45800000000000002,"15min":0.497},"last":0.153},{"interface":"front","average":{"1min":0.214,"5min":0.22800000000000001,"15min":0.246},"min":{"1min":0.16700000000000001,"5min":0.16700000000000001,"15min":0.154},"max":{"1min":0.27800000000000002,"5min":0.432,"15min":0.51100000000000001},"last":0.19600000000000001}]},{"osd":2,"last update":"Fri Oct 6 00:26:53 2023","interfaces":[{"interface":"back","average":{"1min":0.22800000000000001,"5min":0.23999999999999999,"15min":0.23799999999999999},"min":{"1min":0.17899999999999999,"5min":0.16800000000000001,"15min":0.16800000000000001},"max":{"1min":0.30499999999999999,"5min":0.437,"15min":0.46999999999999997},"last":0.186},{"interface":"front","average":{"1min":0.24099999999999999,"5min":0.25700000000000001,"15min":0.25800000000000001},"min":{"1min":0.19900000000000001,"5min":0.19700000000000001,"15min":0.17199999999999999},"max":{"1min":0.316,"5min":0.433,"15min":0.54700000000000004},"last":0.21199999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:27:30.917 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:27:30.917 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:27:30.918 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:27:30.918 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:27:30.918 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:27:30.918 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:27:30.918 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:27:30.919 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:27:30.919 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:27:30.919 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:27:31.233 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:31.231+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:31.728 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:31.727+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:32.265 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:32.264+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:32.771 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:32.770+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:33.234 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:33.233+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:33.763 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:33.761+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:34.232 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:34.230+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:34.801 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:34.800+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:35.220 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:35.218+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:35.823 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:35.822+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:36.235 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:36.233+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:36.811 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:36.809+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:37.261 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:37.260+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:37.788 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:37.786+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:38.239 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:38.238+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:38.800 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:38.799+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:39.195 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:39.193+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:39.829 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:39.828+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:40.213 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:40.211+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:40.797 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:40.796+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:41.246 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:41.244+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:41.807 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:41.805+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:42.279 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:42.278+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:42.764 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:42.762+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:43.251 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:43.250+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:43.737 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:43.736+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:44.257 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:44.255+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:44.709 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:44.708+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:45.230 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:45.229+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:45.754 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:45.752+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:46.188 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:46.188+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:46.706 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:46.706+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:47.140 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:47.140+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:47.696 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:47.696+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:48.128 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:48.127+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:48.679 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:48.678+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:49.155 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:49.154+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:49.701 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:49.700+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:50.164 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:50.163+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:50.717 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:50.717+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:50.919 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:27:51.196 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:51.196+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:51.292 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:27:51.295 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:27:51.307 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":6198,"stamp":"2023-10-06T00:27:50.218538+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001706"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":2061,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:27:47.140001+0000","last_change":"2023-10-06T00:27:47.140001+0000","last_active":"2023-10-06T00:27:47.140001+0000","last_peered":"2023-10-06T00:27:47.140001+0000","last_clean":"2023-10-06T00:27:47.140001+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:27:47.140001+0000","last_undegraded":"2023-10-06T00:27:47.140001+0000","last_fullsized":"2023-10-06T00:27:47.140001+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1964,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:27:44.257301+0000","last_change":"2023-10-06T00:27:44.257301+0000","last_active":"2023-10-06T00:27:44.257301+0000","last_peered":"2023-10-06T00:27:44.257301+0000","last_clean":"2023-10-06T00:27:44.257301+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:27:44.257301+0000","last_undegraded":"2023-10-06T00:27:44.257301+0000","last_fullsized":"2023-10-06T00:27:44.257301+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8621,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:24:42.092671+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:24:42.092671+0000","last_peered":"2023-10-06T00:24:42.092671+0000","last_clean":"2023-10-06T00:24:42.092671+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:24:42.092671+0000","last_undegraded":"2023-10-06T00:24:42.092671+0000","last_fullsized":"2023-10-06T00:24:42.092671+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 647s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11213,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:27:39.194473+0000","last_change":"2023-10-06T00:27:39.194473+0000","last_active":"2023-10-06T00:27:39.194473+0000","last_peered":"2023-10-06T00:27:39.194473+0000","last_clean":"2023-10-06T00:27:39.194473+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:27:39.194473+0000","last_undegraded":"2023-10-06T00:27:39.194473+0000","last_fullsized":"2023-10-06T00:27:39.194473+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2480,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:27:43.251222+0000","last_change":"2023-10-06T00:27:43.251222+0000","last_active":"2023-10-06T00:27:43.251222+0000","last_peered":"2023-10-06T00:27:43.251222+0000","last_clean":"2023-10-06T00:27:43.251222+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:27:43.251222+0000","last_undegraded":"2023-10-06T00:27:43.251222+0000","last_fullsized":"2023-10-06T00:27:43.251222+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11864,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:27:48.127397+0000","last_change":"2023-10-06T00:27:48.127397+0000","last_active":"2023-10-06T00:27:48.127397+0000","last_peered":"2023-10-06T00:27:48.127397+0000","last_clean":"2023-10-06T00:27:48.127397+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:27:48.127397+0000","last_undegraded":"2023-10-06T00:27:48.127397+0000","last_fullsized":"2023-10-06T00:27:48.127397+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11617,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:24:43.565128+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-06T00:24:43.565128+0000","last_peered":"2023-10-06T00:24:43.565128+0000","last_clean":"2023-10-06T00:24:43.565128+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-06T00:24:43.565128+0000","last_undegraded":"2023-10-06T00:24:43.565128+0000","last_fullsized":"2023-10-06T00:24:43.565128+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1787,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:24:43.994138+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-06T00:24:43.994138+0000","last_peered":"2023-10-06T00:24:43.994138+0000","last_clean":"2023-10-06T00:24:43.994138+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-06T00:24:43.994138+0000","last_undegraded":"2023-10-06T00:24:43.994138+0000","last_fullsized":"2023-10-06T00:24:43.994138+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1885,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:27:42.278961+0000","last_change":"2023-10-06T00:27:42.278961+0000","last_active":"2023-10-06T00:27:42.278961+0000","last_peered":"2023-10-06T00:27:42.278961+0000","last_clean":"2023-10-06T00:27:42.278961+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:27:42.278961+0000","last_undegraded":"2023-10-06T00:27:42.278961+0000","last_fullsized":"2023-10-06T00:27:42.278961+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740795,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.218,"5min":0.22800000000000001,"15min":0.22},"min":{"1min":0.16300000000000001,"5min":0.158,"15min":0.13800000000000001},"max":{"1min":0.39800000000000002,"5min":1.093,"15min":1.153},"last":0.187},{"interface":"front","average":{"1min":0.248,"5min":0.245,"15min":0.24099999999999999},"min":{"1min":0.186,"5min":0.18099999999999999,"15min":0.17299999999999999},"max":{"1min":0.41299999999999998,"5min":0.42099999999999999,"15min":1.1659999999999999},"last":0.23400000000000001}]},{"osd":1,"last update":"Fri Oct 6 00:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.251,"5min":0.26500000000000001,"15min":0.25800000000000001},"min":{"1min":0.186,"5min":0.17699999999999999,"15min":0.157},"max":{"1min":0.38200000000000001,"5min":1.196,"15min":1.196},"last":0.20100000000000001},{"interface":"front","average":{"1min":0.27200000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"min":{"1min":0.19400000000000001,"5min":0.19,"15min":0.17999999999999999},"max":{"1min":0.45800000000000002,"5min":0.48199999999999998,"15min":1.2110000000000001},"last":0.27500000000000002}]}]},{"osd":0,"up_from":8,"seq":34359740795,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.19700000000000001,"5min":0.19600000000000001,"15min":0.19900000000000001},"min":{"1min":0.16,"5min":0.13600000000000001,"15min":0.13600000000000001},"max":{"1min":0.35099999999999998,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.20200000000000001},{"interface":"front","average":{"1min":0.26400000000000001,"5min":0.26700000000000002,"15min":0.26900000000000002},"min":{"1min":0.17399999999999999,"5min":0.16500000000000001,"15min":0.16500000000000001},"max":{"1min":0.44800000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.26600000000000001}]},{"osd":2,"last update":"Fri Oct 6 00:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.249,"5min":0.23899999999999999,"15min":0.23999999999999999},"min":{"1min":0.20100000000000001,"5min":0.16800000000000001,"15min":0.16600000000000001},"max":{"1min":0.41699999999999998,"5min":0.5,"15min":0.5},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.30299999999999999,"5min":0.29899999999999999,"15min":0.29999999999999999},"min":{"1min":0.22800000000000001,"5min":0.217,"15min":0.21299999999999999},"max":{"1min":0.47699999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"last":0.30599999999999999}]}]},{"osd":1,"up_from":8,"seq":34359740790,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:26:53 2023","interfaces":[{"interface":"back","average":{"1min":0.191,"5min":0.20999999999999999,"15min":0.21299999999999999},"min":{"1min":0.14000000000000001,"5min":0.14000000000000001,"15min":0.13100000000000001},"max":{"1min":0.29599999999999999,"5min":0.45800000000000002,"15min":0.497},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.214,"5min":0.22800000000000001,"15min":0.246},"min":{"1min":0.16700000000000001,"5min":0.16700000000000001,"15min":0.154},"max":{"1min":0.27800000000000002,"5min":0.432,"15min":0.51100000000000001},"last":0.36899999999999999}]},{"osd":2,"last update":"Fri Oct 6 00:26:53 2023","interfaces":[{"interface":"back","average":{"1min":0.22800000000000001,"5min":0.23999999999999999,"15min":0.23799999999999999},"min":{"1min":0.17899999999999999,"5min":0.16800000000000001,"15min":0.16800000000000001},"max":{"1min":0.30499999999999999,"5min":0.437,"15min":0.46999999999999997},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.24099999999999999,"5min":0.25700000000000001,"15min":0.25800000000000001},"min":{"1min":0.19900000000000001,"5min":0.19700000000000001,"15min":0.17199999999999999},"max":{"1min":0.316,"5min":0.433,"15min":0.54700000000000004},"last":0.436}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:27:51.308 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:27:51.308 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:27:51.309 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:27:51.309 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:27:51.309 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:27:51.309 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:27:51.310 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:27:51.310 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:27:51.310 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:27:51.310 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:27:51.766 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:51.765+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:52.167 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:52.167+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:52.764 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:52.764+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:53.217 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:53.216+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:53.772 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:53.772+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:54.229 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:54.228+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:54.792 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:54.791+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:55.268 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:55.267+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:55.760 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:55.759+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:56.236 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:56.236+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:56.796 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:56.795+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:57.193 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:57.192+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:57.799 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:57.798+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:58.215 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:58.214+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:58.829 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:58.828+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:59.168 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:27:59.167+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:27:59.848 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:27:59.847+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:00.123 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:00.123+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:00.849 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:00.848+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:01.168 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:01.167+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:01.813 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:01.812+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:02.140 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:02.140+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:02.824 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:02.823+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:03.129 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:03.129+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:03.838 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:03.837+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:04.124 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:04.123+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:04.822 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:04.821+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:05.149 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:05.149+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:05.823 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:05.822+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:06.180 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:06.179+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:06.868 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:06.866+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:07.174 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:07.173+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:07.832 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:07.831+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:08.147 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:08.145+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:08.796 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:08.794+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:09.148 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:09.146+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:09.840 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:09.839+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:10.185 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:10.183+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:10.812 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:10.811+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:11.194 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:11.193+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:11.311 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:28:11.690 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:28:11.693 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:28:11.704 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":6208,"stamp":"2023-10-06T00:28:10.222046+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001987"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":2064,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:28:05.148834+0000","last_change":"2023-10-06T00:28:05.148834+0000","last_active":"2023-10-06T00:28:05.148834+0000","last_peered":"2023-10-06T00:28:05.148834+0000","last_clean":"2023-10-06T00:28:05.148834+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:28:05.148834+0000","last_undegraded":"2023-10-06T00:28:05.148834+0000","last_fullsized":"2023-10-06T00:28:05.148834+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1967,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:27:59.167513+0000","last_change":"2023-10-06T00:27:59.167513+0000","last_active":"2023-10-06T00:27:59.167513+0000","last_peered":"2023-10-06T00:27:59.167513+0000","last_clean":"2023-10-06T00:27:59.167513+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:27:59.167513+0000","last_undegraded":"2023-10-06T00:27:59.167513+0000","last_fullsized":"2023-10-06T00:27:59.167513+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8621,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:24:42.092671+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:24:42.092671+0000","last_peered":"2023-10-06T00:24:42.092671+0000","last_clean":"2023-10-06T00:24:42.092671+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:24:42.092671+0000","last_undegraded":"2023-10-06T00:24:42.092671+0000","last_fullsized":"2023-10-06T00:24:42.092671+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 647s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11216,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:27:51.196323+0000","last_change":"2023-10-06T00:27:51.196323+0000","last_active":"2023-10-06T00:27:51.196323+0000","last_peered":"2023-10-06T00:27:51.196323+0000","last_clean":"2023-10-06T00:27:51.196323+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:27:51.196323+0000","last_undegraded":"2023-10-06T00:27:51.196323+0000","last_fullsized":"2023-10-06T00:27:51.196323+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2483,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:27:56.236363+0000","last_change":"2023-10-06T00:27:56.236363+0000","last_active":"2023-10-06T00:27:56.236363+0000","last_peered":"2023-10-06T00:27:56.236363+0000","last_clean":"2023-10-06T00:27:56.236363+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:27:56.236363+0000","last_undegraded":"2023-10-06T00:27:56.236363+0000","last_fullsized":"2023-10-06T00:27:56.236363+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11864,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:27:48.127397+0000","last_change":"2023-10-06T00:27:48.127397+0000","last_active":"2023-10-06T00:27:48.127397+0000","last_peered":"2023-10-06T00:27:48.127397+0000","last_clean":"2023-10-06T00:27:48.127397+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:27:48.127397+0000","last_undegraded":"2023-10-06T00:27:48.127397+0000","last_fullsized":"2023-10-06T00:27:48.127397+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11617,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:24:43.565128+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-06T00:24:43.565128+0000","last_peered":"2023-10-06T00:24:43.565128+0000","last_clean":"2023-10-06T00:24:43.565128+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-06T00:24:43.565128+0000","last_undegraded":"2023-10-06T00:24:43.565128+0000","last_fullsized":"2023-10-06T00:24:43.565128+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1787,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:24:43.994138+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-06T00:24:43.994138+0000","last_peered":"2023-10-06T00:24:43.994138+0000","last_clean":"2023-10-06T00:24:43.994138+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-06T00:24:43.994138+0000","last_undegraded":"2023-10-06T00:24:43.994138+0000","last_fullsized":"2023-10-06T00:24:43.994138+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1888,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:27:53.216186+0000","last_change":"2023-10-06T00:27:53.216186+0000","last_active":"2023-10-06T00:27:53.216186+0000","last_peered":"2023-10-06T00:27:53.216186+0000","last_clean":"2023-10-06T00:27:53.216186+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:27:53.216186+0000","last_undegraded":"2023-10-06T00:27:53.216186+0000","last_fullsized":"2023-10-06T00:27:53.216186+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740799,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.218,"5min":0.22800000000000001,"15min":0.22},"min":{"1min":0.16300000000000001,"5min":0.158,"15min":0.13800000000000001},"max":{"1min":0.39800000000000002,"5min":1.093,"15min":1.153},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.248,"5min":0.245,"15min":0.24099999999999999},"min":{"1min":0.186,"5min":0.18099999999999999,"15min":0.17299999999999999},"max":{"1min":0.41299999999999998,"5min":0.42099999999999999,"15min":1.1659999999999999},"last":0.374}]},{"osd":1,"last update":"Fri Oct 6 00:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.251,"5min":0.26500000000000001,"15min":0.25800000000000001},"min":{"1min":0.186,"5min":0.17699999999999999,"15min":0.157},"max":{"1min":0.38200000000000001,"5min":1.196,"15min":1.196},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.27200000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"min":{"1min":0.19400000000000001,"5min":0.19,"15min":0.17999999999999999},"max":{"1min":0.45800000000000002,"5min":0.48199999999999998,"15min":1.2110000000000001},"last":0.40999999999999998}]}]},{"osd":0,"up_from":8,"seq":34359740799,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.19700000000000001,"5min":0.19600000000000001,"15min":0.19900000000000001},"min":{"1min":0.16,"5min":0.13600000000000001,"15min":0.13600000000000001},"max":{"1min":0.35099999999999998,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.317},{"interface":"front","average":{"1min":0.26400000000000001,"5min":0.26700000000000002,"15min":0.26900000000000002},"min":{"1min":0.17399999999999999,"5min":0.16500000000000001,"15min":0.16500000000000001},"max":{"1min":0.44800000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.29499999999999998}]},{"osd":2,"last update":"Fri Oct 6 00:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.249,"5min":0.23899999999999999,"15min":0.23999999999999999},"min":{"1min":0.20100000000000001,"5min":0.16800000000000001,"15min":0.16600000000000001},"max":{"1min":0.41699999999999998,"5min":0.5,"15min":0.5},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.30299999999999999,"5min":0.29899999999999999,"15min":0.29999999999999999},"min":{"1min":0.22800000000000001,"5min":0.217,"15min":0.21299999999999999},"max":{"1min":0.47699999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"last":0.34899999999999998}]}]},{"osd":1,"up_from":8,"seq":34359740794,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:27:58 2023","interfaces":[{"interface":"back","average":{"1min":0.219,"5min":0.21099999999999999,"15min":0.215},"min":{"1min":0.14299999999999999,"5min":0.14000000000000001,"15min":0.13100000000000001},"max":{"1min":0.40699999999999997,"5min":0.45800000000000002,"15min":0.497},"last":0.16900000000000001},{"interface":"front","average":{"1min":0.23100000000000001,"5min":0.22800000000000001,"15min":0.24299999999999999},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.154},"max":{"1min":0.378,"5min":0.432,"15min":0.51100000000000001},"last":0.19700000000000001}]},{"osd":2,"last update":"Fri Oct 6 00:27:58 2023","interfaces":[{"interface":"back","average":{"1min":0.25,"5min":0.24199999999999999,"15min":0.23999999999999999},"min":{"1min":0.17999999999999999,"5min":0.16800000000000001,"15min":0.16800000000000001},"max":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.46999999999999997},"last":0.216},{"interface":"front","average":{"1min":0.26100000000000001,"5min":0.25700000000000001,"15min":0.25800000000000001},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.17199999999999999},"max":{"1min":0.436,"5min":0.436,"15min":0.54700000000000004},"last":0.22700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:28:11.706 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:28:11.707 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:28:11.707 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:28:11.707 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:28:11.708 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:28:11.708 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:28:11.708 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:28:11.708 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:28:11.709 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:28:11.709 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:28:11.767 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:11.765+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:12.153 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:12.151+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:12.792 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:12.791+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:13.180 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:13.178+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:13.751 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:13.750+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:14.204 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:14.203+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:14.782 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:14.781+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:15.254 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:15.253+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:15.806 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:15.804+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:16.299 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:16.298+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:16.844 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:16.843+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:17.260 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:17.259+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:17.869 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:17.867+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:18.286 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:18.285+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:18.872 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:18.871+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:19.280 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:19.278+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:19.870 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:19.869+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:20.307 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:20.305+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:20.841 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:20.840+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:21.269 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:21.267+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:21.840 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:21.838+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:22.288 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:22.286+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:22.835 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:22.833+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:23.294 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:23.293+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:23.810 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:23.809+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:24.341 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:24.340+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:24.840 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:24.838+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:25.302 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:25.301+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:25.820 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:25.819+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:26.350 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:26.349+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:26.860 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:26.859+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:27.368 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:27.367+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:27.876 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:27.875+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:28.408 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:28.406+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:28.925 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:28.923+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:29.432 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:29.430+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:29.887 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:29.885+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:30.457 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:30.456+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:30.857 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:30.855+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:31.477 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:31.475+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:31.710 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:28:31.853 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:31.851+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:32.050 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:28:32.052 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:28:32.064 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":6218,"stamp":"2023-10-06T00:28:30.225189+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001954"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":2067,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:28:22.287526+0000","last_change":"2023-10-06T00:28:22.287526+0000","last_active":"2023-10-06T00:28:22.287526+0000","last_peered":"2023-10-06T00:28:22.287526+0000","last_clean":"2023-10-06T00:28:22.287526+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:28:22.287526+0000","last_undegraded":"2023-10-06T00:28:22.287526+0000","last_fullsized":"2023-10-06T00:28:22.287526+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1970,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:28:21.268680+0000","last_change":"2023-10-06T00:28:21.268680+0000","last_active":"2023-10-06T00:28:21.268680+0000","last_peered":"2023-10-06T00:28:21.268680+0000","last_clean":"2023-10-06T00:28:21.268680+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:28:21.268680+0000","last_undegraded":"2023-10-06T00:28:21.268680+0000","last_fullsized":"2023-10-06T00:28:21.268680+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8621,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:24:42.092671+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:24:42.092671+0000","last_peered":"2023-10-06T00:24:42.092671+0000","last_clean":"2023-10-06T00:24:42.092671+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:24:42.092671+0000","last_undegraded":"2023-10-06T00:24:42.092671+0000","last_fullsized":"2023-10-06T00:24:42.092671+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 647s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11222,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:28:26.350177+0000","last_change":"2023-10-06T00:28:26.350177+0000","last_active":"2023-10-06T00:28:26.350177+0000","last_peered":"2023-10-06T00:28:26.350177+0000","last_clean":"2023-10-06T00:28:26.350177+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:28:26.350177+0000","last_undegraded":"2023-10-06T00:28:26.350177+0000","last_fullsized":"2023-10-06T00:28:26.350177+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2489,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:28:28.407766+0000","last_change":"2023-10-06T00:28:28.407766+0000","last_active":"2023-10-06T00:28:28.407766+0000","last_peered":"2023-10-06T00:28:28.407766+0000","last_clean":"2023-10-06T00:28:28.407766+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:28:28.407766+0000","last_undegraded":"2023-10-06T00:28:28.407766+0000","last_fullsized":"2023-10-06T00:28:28.407766+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11870,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:28:23.294154+0000","last_change":"2023-10-06T00:28:23.294154+0000","last_active":"2023-10-06T00:28:23.294154+0000","last_peered":"2023-10-06T00:28:23.294154+0000","last_clean":"2023-10-06T00:28:23.294154+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:28:23.294154+0000","last_undegraded":"2023-10-06T00:28:23.294154+0000","last_fullsized":"2023-10-06T00:28:23.294154+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11617,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:24:43.565128+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-06T00:24:43.565128+0000","last_peered":"2023-10-06T00:24:43.565128+0000","last_clean":"2023-10-06T00:24:43.565128+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-06T00:24:43.565128+0000","last_undegraded":"2023-10-06T00:24:43.565128+0000","last_fullsized":"2023-10-06T00:24:43.565128+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1787,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:24:43.994138+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-06T00:24:43.994138+0000","last_peered":"2023-10-06T00:24:43.994138+0000","last_clean":"2023-10-06T00:24:43.994138+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-06T00:24:43.994138+0000","last_undegraded":"2023-10-06T00:24:43.994138+0000","last_fullsized":"2023-10-06T00:24:43.994138+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1894,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:28:27.368231+0000","last_change":"2023-10-06T00:28:27.368231+0000","last_active":"2023-10-06T00:28:27.368231+0000","last_peered":"2023-10-06T00:28:27.368231+0000","last_clean":"2023-10-06T00:28:27.368231+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:28:27.368231+0000","last_undegraded":"2023-10-06T00:28:27.368231+0000","last_fullsized":"2023-10-06T00:28:27.368231+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740803,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.218,"5min":0.22800000000000001,"15min":0.22},"min":{"1min":0.16300000000000001,"5min":0.158,"15min":0.13800000000000001},"max":{"1min":0.39800000000000002,"5min":1.093,"15min":1.153},"last":0.20100000000000001},{"interface":"front","average":{"1min":0.248,"5min":0.245,"15min":0.24099999999999999},"min":{"1min":0.186,"5min":0.18099999999999999,"15min":0.17299999999999999},"max":{"1min":0.41299999999999998,"5min":0.42099999999999999,"15min":1.1659999999999999},"last":0.23699999999999999}]},{"osd":1,"last update":"Fri Oct 6 00:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.27500000000000002,"15min":0.26300000000000001},"min":{"1min":0.19600000000000001,"5min":0.186,"15min":0.157},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.222},{"interface":"front","average":{"1min":0.29499999999999998,"5min":0.28100000000000003,"15min":0.28000000000000003},"min":{"1min":0.222,"5min":0.19,"15min":0.17999999999999999},"max":{"1min":0.45000000000000001,"5min":0.48199999999999998,"15min":1.2110000000000001},"last":0.27000000000000002}]}]},{"osd":0,"up_from":8,"seq":34359740803,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.22,"5min":0.20000000000000001,"15min":0.20200000000000001},"min":{"1min":0.14899999999999999,"5min":0.13600000000000001,"15min":0.13600000000000001},"max":{"1min":0.32600000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.25600000000000001,"5min":0.26500000000000001,"15min":0.26900000000000002},"min":{"1min":0.187,"5min":0.16500000000000001,"15min":0.16500000000000001},"max":{"1min":0.377,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.377}]},{"osd":2,"last update":"Fri Oct 6 00:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.23699999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"min":{"1min":0.17899999999999999,"5min":0.16800000000000001,"15min":0.16600000000000001},"max":{"1min":0.35999999999999999,"5min":0.5,"15min":0.5},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.29099999999999998,"5min":0.29799999999999999,"15min":0.29999999999999999},"min":{"1min":0.23400000000000001,"5min":0.217,"15min":0.21299999999999999},"max":{"1min":0.39700000000000002,"5min":0.53800000000000003,"15min":0.53800000000000003},"last":0.39700000000000002}]}]},{"osd":1,"up_from":8,"seq":34359740798,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:27:58 2023","interfaces":[{"interface":"back","average":{"1min":0.219,"5min":0.21099999999999999,"15min":0.215},"min":{"1min":0.14299999999999999,"5min":0.14000000000000001,"15min":0.13100000000000001},"max":{"1min":0.40699999999999997,"5min":0.45800000000000002,"15min":0.497},"last":0.186},{"interface":"front","average":{"1min":0.23100000000000001,"5min":0.22800000000000001,"15min":0.24299999999999999},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.154},"max":{"1min":0.378,"5min":0.432,"15min":0.51100000000000001},"last":0.16300000000000001}]},{"osd":2,"last update":"Fri Oct 6 00:27:58 2023","interfaces":[{"interface":"back","average":{"1min":0.25,"5min":0.24199999999999999,"15min":0.23999999999999999},"min":{"1min":0.17999999999999999,"5min":0.16800000000000001,"15min":0.16800000000000001},"max":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.46999999999999997},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.26100000000000001,"5min":0.25700000000000001,"15min":0.25800000000000001},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.17199999999999999},"max":{"1min":0.436,"5min":0.436,"15min":0.54700000000000004},"last":0.23999999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:28:32.066 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:28:32.066 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:28:32.066 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:28:32.067 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:28:32.067 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:28:32.067 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:28:32.067 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:28:32.067 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:28:32.067 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:28:32.068 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:28:32.508 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:32.506+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:32.886 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:32.884+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:33.479 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:33.477+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:33.870 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:33.869+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:34.483 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:34.481+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:34.864 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:34.862+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:35.505 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:35.504+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:35.886 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:35.884+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:36.506 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:36.504+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:36.887 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:36.886+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:37.497 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:37.495+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:37.935 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:37.933+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:38.487 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:38.486+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:38.984 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:38.983+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:39.494 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:39.493+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:40.010 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:40.008+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:40.456 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:40.455+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:41.049 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:41.048+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:41.444 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:41.442+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:42.004 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:42.003+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:42.421 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:42.419+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:43.036 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:43.035+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:43.436 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:43.435+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:44.033 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:44.033+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:44.417 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:44.416+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:45.052 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:45.052+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:45.389 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:45.386+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:46.082 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:46.081+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:46.340 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:46.339+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:47.041 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:47.039+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:47.352 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:47.351+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:48.010 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:48.010+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:48.383 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:48.383+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:49.034 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:49.034+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:49.340 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:49.339+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:50.039 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:50.038+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:50.335 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:50.335+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:51.008 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:51.008+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:51.310 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:51.310+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:52.023 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:52.023+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:52.069 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:28:52.315 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:52.315+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:52.408 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:28:52.410 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:28:52.421 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":6229,"stamp":"2023-10-06T00:28:52.228883+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002189"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":2073,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:28:48.383562+0000","last_change":"2023-10-06T00:28:48.383562+0000","last_active":"2023-10-06T00:28:48.383562+0000","last_peered":"2023-10-06T00:28:48.383562+0000","last_clean":"2023-10-06T00:28:48.383562+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:28:48.383562+0000","last_undegraded":"2023-10-06T00:28:48.383562+0000","last_fullsized":"2023-10-06T00:28:48.383562+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1976,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:28:47.352031+0000","last_change":"2023-10-06T00:28:47.352031+0000","last_active":"2023-10-06T00:28:47.352031+0000","last_peered":"2023-10-06T00:28:47.352031+0000","last_clean":"2023-10-06T00:28:47.352031+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:28:47.352031+0000","last_undegraded":"2023-10-06T00:28:47.352031+0000","last_fullsized":"2023-10-06T00:28:47.352031+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8621,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:24:42.092671+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:24:42.092671+0000","last_peered":"2023-10-06T00:24:42.092671+0000","last_clean":"2023-10-06T00:24:42.092671+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:24:42.092671+0000","last_undegraded":"2023-10-06T00:24:42.092671+0000","last_fullsized":"2023-10-06T00:24:42.092671+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 647s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11225,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:28:36.505942+0000","last_change":"2023-10-06T00:28:36.505942+0000","last_active":"2023-10-06T00:28:36.505942+0000","last_peered":"2023-10-06T00:28:36.505942+0000","last_clean":"2023-10-06T00:28:36.505942+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:28:36.505942+0000","last_undegraded":"2023-10-06T00:28:36.505942+0000","last_fullsized":"2023-10-06T00:28:36.505942+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2492,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:28:43.435912+0000","last_change":"2023-10-06T00:28:43.435912+0000","last_active":"2023-10-06T00:28:43.435912+0000","last_peered":"2023-10-06T00:28:43.435912+0000","last_clean":"2023-10-06T00:28:43.435912+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:28:43.435912+0000","last_undegraded":"2023-10-06T00:28:43.435912+0000","last_fullsized":"2023-10-06T00:28:43.435912+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:28:33.478693+0000","last_change":"2023-10-06T00:28:33.478693+0000","last_active":"2023-10-06T00:28:33.478693+0000","last_peered":"2023-10-06T00:28:33.478693+0000","last_clean":"2023-10-06T00:28:33.478693+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:28:33.478693+0000","last_undegraded":"2023-10-06T00:28:33.478693+0000","last_fullsized":"2023-10-06T00:28:33.478693+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11617,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:24:43.565128+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-06T00:24:43.565128+0000","last_peered":"2023-10-06T00:24:43.565128+0000","last_clean":"2023-10-06T00:24:43.565128+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-06T00:24:43.565128+0000","last_undegraded":"2023-10-06T00:24:43.565128+0000","last_fullsized":"2023-10-06T00:24:43.565128+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1787,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:24:43.994138+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-06T00:24:43.994138+0000","last_peered":"2023-10-06T00:24:43.994138+0000","last_clean":"2023-10-06T00:24:43.994138+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-06T00:24:43.994138+0000","last_undegraded":"2023-10-06T00:24:43.994138+0000","last_fullsized":"2023-10-06T00:24:43.994138+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1897,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:28:42.420905+0000","last_change":"2023-10-06T00:28:42.420905+0000","last_active":"2023-10-06T00:28:42.420905+0000","last_peered":"2023-10-06T00:28:42.420905+0000","last_clean":"2023-10-06T00:28:42.420905+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:28:42.420905+0000","last_undegraded":"2023-10-06T00:28:42.420905+0000","last_fullsized":"2023-10-06T00:28:42.420905+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740807,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.26500000000000001,"5min":0.23400000000000001,"15min":0.224},"min":{"1min":0.17000000000000001,"5min":0.159,"15min":0.13800000000000001},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.19900000000000001},{"interface":"front","average":{"1min":0.26200000000000001,"5min":0.246,"15min":0.24299999999999999},"min":{"1min":0.19400000000000001,"5min":0.18099999999999999,"15min":0.17299999999999999},"max":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":1.1659999999999999},"last":0.224}]},{"osd":1,"last update":"Fri Oct 6 00:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.27500000000000002,"15min":0.26300000000000001},"min":{"1min":0.19600000000000001,"5min":0.186,"15min":0.157},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.29499999999999998,"5min":0.28100000000000003,"15min":0.28000000000000003},"min":{"1min":0.222,"5min":0.19,"15min":0.17999999999999999},"max":{"1min":0.45000000000000001,"5min":0.48199999999999998,"15min":1.2110000000000001},"last":0.26800000000000002}]}]},{"osd":0,"up_from":8,"seq":34359740807,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.22,"5min":0.20000000000000001,"15min":0.20200000000000001},"min":{"1min":0.14899999999999999,"5min":0.13600000000000001,"15min":0.13600000000000001},"max":{"1min":0.32600000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.187},{"interface":"front","average":{"1min":0.25600000000000001,"5min":0.26500000000000001,"15min":0.26900000000000002},"min":{"1min":0.187,"5min":0.16500000000000001,"15min":0.16500000000000001},"max":{"1min":0.377,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.26200000000000001}]},{"osd":2,"last update":"Fri Oct 6 00:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.23699999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"min":{"1min":0.17899999999999999,"5min":0.16800000000000001,"15min":0.16600000000000001},"max":{"1min":0.35999999999999999,"5min":0.5,"15min":0.5},"last":0.223},{"interface":"front","average":{"1min":0.29099999999999998,"5min":0.29799999999999999,"15min":0.29999999999999999},"min":{"1min":0.23400000000000001,"5min":0.217,"15min":0.21299999999999999},"max":{"1min":0.39700000000000002,"5min":0.53800000000000003,"15min":0.53800000000000003},"last":0.28100000000000003}]}]},{"osd":1,"up_from":8,"seq":34359740802,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:27:58 2023","interfaces":[{"interface":"back","average":{"1min":0.219,"5min":0.21099999999999999,"15min":0.215},"min":{"1min":0.14299999999999999,"5min":0.14000000000000001,"15min":0.13100000000000001},"max":{"1min":0.40699999999999997,"5min":0.45800000000000002,"15min":0.497},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.23100000000000001,"5min":0.22800000000000001,"15min":0.24299999999999999},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.154},"max":{"1min":0.378,"5min":0.432,"15min":0.51100000000000001},"last":0.19800000000000001}]},{"osd":2,"last update":"Fri Oct 6 00:27:58 2023","interfaces":[{"interface":"back","average":{"1min":0.25,"5min":0.24199999999999999,"15min":0.23999999999999999},"min":{"1min":0.17999999999999999,"5min":0.16800000000000001,"15min":0.16800000000000001},"max":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.46999999999999997},"last":0.217},{"interface":"front","average":{"1min":0.26100000000000001,"5min":0.25700000000000001,"15min":0.25800000000000001},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.17199999999999999},"max":{"1min":0.436,"5min":0.436,"15min":0.54700000000000004},"last":0.25900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:28:52.422 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:28:52.423 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:28:52.423 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:28:52.423 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:28:52.423 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:28:52.424 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:28:52.424 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:28:52.424 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:28:52.424 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:28:52.424 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.7 2023-10-06T00:28:52.802 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 2.7 on osd.1 to deep-scrub 2023-10-06T00:28:52.814 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.6 2023-10-06T00:28:53.050 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:53.049+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:53.197 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 2.6 on osd.1 to deep-scrub 2023-10-06T00:28:53.208 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.1 2023-10-06T00:28:53.346 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:53.345+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:53.555 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 3.1 on osd.0 to deep-scrub 2023-10-06T00:28:53.567 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.2 2023-10-06T00:28:53.936 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 3.2 on osd.1 to deep-scrub 2023-10-06T00:28:53.948 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.3 2023-10-06T00:28:54.063 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:54.063+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:54.276 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 2.3 on osd.1 to deep-scrub 2023-10-06T00:28:54.287 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.0 2023-10-06T00:28:54.321 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:54.320+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:54.660 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 3.0 on osd.1 to deep-scrub 2023-10-06T00:28:54.672 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.3 2023-10-06T00:28:55.041 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 3.3 on osd.0 to deep-scrub 2023-10-06T00:28:55.052 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.2 2023-10-06T00:28:55.087 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:55.087+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:55.286 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:55.285+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:55.436 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 2.2 on osd.0 to deep-scrub 2023-10-06T00:28:55.448 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.4 2023-10-06T00:28:55.848 INFO:teuthology.orchestra.run.smithi167.stderr:instructing pg 2.4 on osd.1 to deep-scrub 2023-10-06T00:28:55.859 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:28:56.053 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:56.051+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:56.288 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:56.287+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:57.029 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:57.029+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:57.249 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:57.249+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:58.054 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:58.053+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:58.296 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:58.296+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:59.018 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:28:59.018+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:28:59.289 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:28:59.289+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:00.055 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:00.054+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:00.301 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:00.301+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:01.082 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:01.081+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:01.320 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:01.319+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:02.077 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:02.076+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:02.347 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:02.346+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:03.103 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:03.102+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:03.365 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:03.363+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:04.133 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:04.133+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:04.399 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:04.398+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:05.174 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:05.173+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:05.376 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:05.375+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:06.173 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:06.171+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:06.373 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:06.372+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:07.160 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:07.158+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:07.328 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:07.326+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:08.168 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:08.166+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:08.339 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:08.337+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:09.198 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:09.196+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:09.313 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:09.311+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:10.149 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:10.147+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:10.319 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:10.318+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:11.199 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:11.198+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:11.364 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:11.362+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:12.182 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:12.180+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:12.409 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:12.407+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:13.170 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:13.168+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:13.406 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:13.404+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:14.172 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:14.170+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:14.368 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:14.366+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:15.138 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:15.137+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:15.373 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:15.371+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:15.861 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:29:16.117 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:16.115+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:16.209 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:29:16.212 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:29:16.222 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":6240,"stamp":"2023-10-06T00:29:14.232410+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001774"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":2077,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:29:07.327934+0000","last_change":"2023-10-06T00:29:07.327934+0000","last_active":"2023-10-06T00:29:07.327934+0000","last_peered":"2023-10-06T00:29:07.327934+0000","last_clean":"2023-10-06T00:29:07.327934+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:29:07.327934+0000","last_undegraded":"2023-10-06T00:29:07.327934+0000","last_fullsized":"2023-10-06T00:29:07.327934+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1980,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:29:00.300859+0000","last_change":"2023-10-06T00:29:00.300859+0000","last_active":"2023-10-06T00:29:00.300859+0000","last_peered":"2023-10-06T00:29:00.300859+0000","last_clean":"2023-10-06T00:29:00.300859+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:29:00.300859+0000","last_undegraded":"2023-10-06T00:29:00.300859+0000","last_fullsized":"2023-10-06T00:29:00.300859+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8622,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:26:47.872288+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:26:47.872288+0000","last_peered":"2023-10-06T00:26:47.872288+0000","last_clean":"2023-10-06T00:26:47.872288+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:26:47.872288+0000","last_undegraded":"2023-10-06T00:26:47.872288+0000","last_fullsized":"2023-10-06T00:26:47.872288+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 773s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11232,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:29:10.319018+0000","last_change":"2023-10-06T00:29:10.319018+0000","last_active":"2023-10-06T00:29:10.319018+0000","last_peered":"2023-10-06T00:29:10.319018+0000","last_clean":"2023-10-06T00:29:10.319018+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:29:10.319018+0000","last_undegraded":"2023-10-06T00:29:10.319018+0000","last_fullsized":"2023-10-06T00:29:10.319018+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2496,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:29:01.319904+0000","last_change":"2023-10-06T00:29:01.319904+0000","last_active":"2023-10-06T00:29:01.319904+0000","last_peered":"2023-10-06T00:29:01.319904+0000","last_clean":"2023-10-06T00:29:01.319904+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:29:01.319904+0000","last_undegraded":"2023-10-06T00:29:01.319904+0000","last_fullsized":"2023-10-06T00:29:01.319904+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11880,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:29:09.313091+0000","last_change":"2023-10-06T00:29:09.313091+0000","last_active":"2023-10-06T00:29:09.313091+0000","last_peered":"2023-10-06T00:29:09.313091+0000","last_clean":"2023-10-06T00:29:09.313091+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:29:09.313091+0000","last_undegraded":"2023-10-06T00:29:09.313091+0000","last_fullsized":"2023-10-06T00:29:09.313091+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11618,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:26:49.384875+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-06T00:26:49.384875+0000","last_peered":"2023-10-06T00:26:49.384875+0000","last_clean":"2023-10-06T00:26:49.384875+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-06T00:26:49.384875+0000","last_undegraded":"2023-10-06T00:26:49.384875+0000","last_fullsized":"2023-10-06T00:26:49.384875+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1788,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:26:49.777242+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-06T00:26:49.777242+0000","last_peered":"2023-10-06T00:26:49.777242+0000","last_clean":"2023-10-06T00:26:49.777242+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-06T00:26:49.777242+0000","last_undegraded":"2023-10-06T00:26:49.777242+0000","last_fullsized":"2023-10-06T00:26:49.777242+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1904,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:29:12.408555+0000","last_change":"2023-10-06T00:29:12.408555+0000","last_active":"2023-10-06T00:29:12.408555+0000","last_peered":"2023-10-06T00:29:12.408555+0000","last_clean":"2023-10-06T00:29:12.408555+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:29:12.408555+0000","last_undegraded":"2023-10-06T00:29:12.408555+0000","last_fullsized":"2023-10-06T00:29:12.408555+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740811,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:28:31 2023","interfaces":[{"interface":"back","average":{"1min":0.26500000000000001,"5min":0.23400000000000001,"15min":0.224},"min":{"1min":0.17000000000000001,"5min":0.159,"15min":0.13800000000000001},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.26200000000000001,"5min":0.246,"15min":0.24299999999999999},"min":{"1min":0.19400000000000001,"5min":0.18099999999999999,"15min":0.17299999999999999},"max":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":1.1659999999999999},"last":0.23400000000000001}]},{"osd":1,"last update":"Fri Oct 6 00:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.27500000000000002,"15min":0.26300000000000001},"min":{"1min":0.19600000000000001,"5min":0.186,"15min":0.157},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.253},{"interface":"front","average":{"1min":0.29499999999999998,"5min":0.28100000000000003,"15min":0.28000000000000003},"min":{"1min":0.222,"5min":0.19,"15min":0.17999999999999999},"max":{"1min":0.45000000000000001,"5min":0.48199999999999998,"15min":1.2110000000000001},"last":0.28299999999999997}]}]},{"osd":0,"up_from":8,"seq":34359740812,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.22,"5min":0.20000000000000001,"15min":0.20200000000000001},"min":{"1min":0.14899999999999999,"5min":0.13600000000000001,"15min":0.13600000000000001},"max":{"1min":0.32600000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.17100000000000001},{"interface":"front","average":{"1min":0.25600000000000001,"5min":0.26500000000000001,"15min":0.26900000000000002},"min":{"1min":0.187,"5min":0.16500000000000001,"15min":0.16500000000000001},"max":{"1min":0.377,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.23000000000000001}]},{"osd":2,"last update":"Fri Oct 6 00:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.23699999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"min":{"1min":0.17899999999999999,"5min":0.16800000000000001,"15min":0.16600000000000001},"max":{"1min":0.35999999999999999,"5min":0.5,"15min":0.5},"last":0.20599999999999999},{"interface":"front","average":{"1min":0.29099999999999998,"5min":0.29799999999999999,"15min":0.29999999999999999},"min":{"1min":0.23400000000000001,"5min":0.217,"15min":0.21299999999999999},"max":{"1min":0.39700000000000002,"5min":0.53800000000000003,"15min":0.53800000000000003},"last":0.27300000000000002}]}]},{"osd":1,"up_from":8,"seq":34359740807,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.215,"5min":0.20999999999999999,"15min":0.215},"min":{"1min":0.14799999999999999,"5min":0.14000000000000001,"15min":0.13100000000000001},"max":{"1min":0.36299999999999999,"5min":0.45800000000000002,"15min":0.497},"last":0.16700000000000001},{"interface":"front","average":{"1min":0.23400000000000001,"5min":0.22700000000000001,"15min":0.24099999999999999},"min":{"1min":0.16300000000000001,"5min":0.16300000000000001,"15min":0.154},"max":{"1min":0.39800000000000002,"5min":0.432,"15min":0.51100000000000001},"last":0.192}]},{"osd":2,"last update":"Fri Oct 6 00:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.247,"5min":0.23999999999999999,"15min":0.23999999999999999},"min":{"1min":0.182,"5min":0.16800000000000001,"15min":0.16800000000000001},"max":{"1min":0.41699999999999998,"5min":0.45100000000000001,"15min":0.46999999999999997},"last":0.23899999999999999},{"interface":"front","average":{"1min":0.25900000000000001,"5min":0.254,"15min":0.25700000000000001},"min":{"1min":0.20499999999999999,"5min":0.19700000000000001,"15min":0.17199999999999999},"max":{"1min":0.436,"5min":0.436,"15min":0.54700000000000004},"last":0.20499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:29:16.224 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:29:16.224 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:29:16.224 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:29:16.225 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:29:16.225 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:29:16.225 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:29:16.225 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:29:16.225 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:29:16.226 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:29:16.226 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:29:16.327 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:16.326+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:17.115 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:17.113+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:17.320 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:17.318+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:18.099 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:18.098+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:18.320 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:18.318+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:19.137 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:19.136+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:19.297 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:19.295+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:20.093 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:20.092+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:20.300 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:20.298+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:21.141 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:21.140+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:21.324 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:21.323+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:22.109 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:22.108+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:22.355 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:22.353+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:23.136 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:23.134+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:23.361 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:23.359+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:24.140 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:24.138+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:24.329 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:24.328+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:25.175 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:25.174+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:25.289 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:25.288+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:26.153 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:26.152+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:26.286 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:26.284+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:27.155 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:27.154+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:27.318 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:27.317+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:28.117 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:28.115+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:28.314 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:28.312+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:29.113 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:29.111+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:29.265 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:29.263+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:30.092 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:30.091+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:30.315 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:30.314+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:31.142 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:31.141+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:31.278 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:31.277+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:32.134 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:32.132+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:32.249 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:32.248+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:33.139 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:33.137+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:33.254 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:33.253+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:34.101 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:34.100+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:34.247 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:34.246+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:35.073 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:35.072+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:35.289 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:35.287+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:36.025 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:36.024+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:36.226 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:29:36.270 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:36.267+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:36.610 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:29:36.613 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:29:36.623 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":6251,"stamp":"2023-10-06T00:29:36.235853+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001979"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":2080,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:29:22.354563+0000","last_change":"2023-10-06T00:29:22.354563+0000","last_active":"2023-10-06T00:29:22.354563+0000","last_peered":"2023-10-06T00:29:22.354563+0000","last_clean":"2023-10-06T00:29:22.354563+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:29:22.354563+0000","last_undegraded":"2023-10-06T00:29:22.354563+0000","last_fullsized":"2023-10-06T00:29:22.354563+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1986,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:29:32.249345+0000","last_change":"2023-10-06T00:29:32.249345+0000","last_active":"2023-10-06T00:29:32.249345+0000","last_peered":"2023-10-06T00:29:32.249345+0000","last_clean":"2023-10-06T00:29:32.249345+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:29:32.249345+0000","last_undegraded":"2023-10-06T00:29:32.249345+0000","last_fullsized":"2023-10-06T00:29:32.249345+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8622,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:26:47.872288+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:26:47.872288+0000","last_peered":"2023-10-06T00:26:47.872288+0000","last_clean":"2023-10-06T00:26:47.872288+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:26:47.872288+0000","last_undegraded":"2023-10-06T00:26:47.872288+0000","last_fullsized":"2023-10-06T00:26:47.872288+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 773s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11235,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:29:30.315058+0000","last_change":"2023-10-06T00:29:30.315058+0000","last_active":"2023-10-06T00:29:30.315058+0000","last_peered":"2023-10-06T00:29:30.315058+0000","last_clean":"2023-10-06T00:29:30.315058+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:29:30.315058+0000","last_undegraded":"2023-10-06T00:29:30.315058+0000","last_fullsized":"2023-10-06T00:29:30.315058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2499,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:29:20.299518+0000","last_change":"2023-10-06T00:29:20.299518+0000","last_active":"2023-10-06T00:29:20.299518+0000","last_peered":"2023-10-06T00:29:20.299518+0000","last_clean":"2023-10-06T00:29:20.299518+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:29:20.299518+0000","last_undegraded":"2023-10-06T00:29:20.299518+0000","last_fullsized":"2023-10-06T00:29:20.299518+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11883,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:29:23.360861+0000","last_change":"2023-10-06T00:29:23.360861+0000","last_active":"2023-10-06T00:29:23.360861+0000","last_peered":"2023-10-06T00:29:23.360861+0000","last_clean":"2023-10-06T00:29:23.360861+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:29:23.360861+0000","last_undegraded":"2023-10-06T00:29:23.360861+0000","last_fullsized":"2023-10-06T00:29:23.360861+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11618,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:26:49.384875+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-06T00:26:49.384875+0000","last_peered":"2023-10-06T00:26:49.384875+0000","last_clean":"2023-10-06T00:26:49.384875+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-06T00:26:49.384875+0000","last_undegraded":"2023-10-06T00:26:49.384875+0000","last_fullsized":"2023-10-06T00:26:49.384875+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1788,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:26:49.777242+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-06T00:26:49.777242+0000","last_peered":"2023-10-06T00:26:49.777242+0000","last_clean":"2023-10-06T00:26:49.777242+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-06T00:26:49.777242+0000","last_undegraded":"2023-10-06T00:26:49.777242+0000","last_fullsized":"2023-10-06T00:26:49.777242+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1907,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:29:31.278254+0000","last_change":"2023-10-06T00:29:31.278254+0000","last_active":"2023-10-06T00:29:31.278254+0000","last_peered":"2023-10-06T00:29:31.278254+0000","last_clean":"2023-10-06T00:29:31.278254+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:29:31.278254+0000","last_undegraded":"2023-10-06T00:29:31.278254+0000","last_fullsized":"2023-10-06T00:29:31.278254+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740816,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:29:33 2023","interfaces":[{"interface":"back","average":{"1min":0.23699999999999999,"5min":0.23799999999999999,"15min":0.22500000000000001},"min":{"1min":0.16300000000000001,"5min":0.159,"15min":0.13800000000000001},"max":{"1min":0.44900000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.318},{"interface":"front","average":{"1min":0.25700000000000001,"5min":0.249,"15min":0.24399999999999999},"min":{"1min":0.186,"5min":0.18099999999999999,"15min":0.17299999999999999},"max":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":1.1659999999999999},"last":0.27800000000000002}]},{"osd":1,"last update":"Fri Oct 6 00:29:29 2023","interfaces":[{"interface":"back","average":{"1min":0.27400000000000002,"5min":0.27900000000000003,"15min":0.26500000000000001},"min":{"1min":0.19900000000000001,"5min":0.186,"15min":0.157},"max":{"1min":0.44600000000000001,"5min":1.198,"15min":1.198},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.28499999999999998,"5min":0.28100000000000003,"15min":0.28000000000000003},"min":{"1min":0.217,"5min":0.19,"15min":0.17999999999999999},"max":{"1min":0.46000000000000002,"5min":0.48199999999999998,"15min":1.2110000000000001},"last":0.33300000000000002}]}]},{"osd":0,"up_from":8,"seq":34359740816,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.215,"5min":0.20499999999999999,"15min":0.20100000000000001},"min":{"1min":0.14199999999999999,"5min":0.13600000000000001,"15min":0.13600000000000001},"max":{"1min":0.40500000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.26100000000000001,"5min":0.26500000000000001,"15min":0.26700000000000002},"min":{"1min":0.20000000000000001,"5min":0.16500000000000001,"15min":0.16500000000000001},"max":{"1min":0.44700000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.28999999999999998}]},{"osd":2,"last update":"Fri Oct 6 00:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.245,"5min":0.24299999999999999,"15min":0.23899999999999999},"min":{"1min":0.16800000000000001,"5min":0.16800000000000001,"15min":0.16600000000000001},"max":{"1min":0.39000000000000001,"5min":0.5,"15min":0.5},"last":0.255},{"interface":"front","average":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29799999999999999},"min":{"1min":0.22900000000000001,"5min":0.217,"15min":0.21299999999999999},"max":{"1min":0.442,"5min":0.53800000000000003,"15min":0.53800000000000003},"last":0.33600000000000002}]}]},{"osd":1,"up_from":8,"seq":34359740811,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.215,"5min":0.20999999999999999,"15min":0.215},"min":{"1min":0.14799999999999999,"5min":0.14000000000000001,"15min":0.13100000000000001},"max":{"1min":0.36299999999999999,"5min":0.45800000000000002,"15min":0.497},"last":0.16300000000000001},{"interface":"front","average":{"1min":0.23400000000000001,"5min":0.22700000000000001,"15min":0.24099999999999999},"min":{"1min":0.16300000000000001,"5min":0.16300000000000001,"15min":0.154},"max":{"1min":0.39800000000000002,"5min":0.432,"15min":0.51100000000000001},"last":0.19500000000000001}]},{"osd":2,"last update":"Fri Oct 6 00:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.247,"5min":0.23999999999999999,"15min":0.23999999999999999},"min":{"1min":0.182,"5min":0.16800000000000001,"15min":0.16800000000000001},"max":{"1min":0.41699999999999998,"5min":0.45100000000000001,"15min":0.46999999999999997},"last":0.214},{"interface":"front","average":{"1min":0.25900000000000001,"5min":0.254,"15min":0.25700000000000001},"min":{"1min":0.20499999999999999,"5min":0.19700000000000001,"15min":0.17199999999999999},"max":{"1min":0.436,"5min":0.436,"15min":0.54700000000000004},"last":0.222}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:29:36.625 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:29:36.625 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:29:36.626 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:29:36.626 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:29:36.627 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:29:36.627 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:29:36.627 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:29:36.627 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:29:36.627 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:29:36.628 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:29:37.067 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:37.065+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:37.282 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:37.280+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:38.040 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:38.038+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:38.274 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:38.272+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:39.030 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:39.028+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:39.273 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:39.272+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:40.047 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:40.045+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:40.275 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:40.274+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:41.030 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:41.029+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:41.232 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:41.231+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:42.011 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:42.009+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:42.241 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:42.241+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:42.976 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:42.975+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:43.242 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:43.241+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:43.969 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:43.969+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:44.193 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:44.193+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:45.000 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:44.999+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:45.218 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:45.217+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:45.995 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:45.995+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:46.213 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:46.212+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:47.005 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:47.004+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:47.190 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:47.190+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:47.966 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:47.965+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:48.173 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:48.172+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:48.952 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:48.952+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:49.200 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:49.199+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:49.911 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:49.910+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:50.245 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:50.245+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:50.960 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:50.959+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:51.224 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:51.224+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:51.915 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:51.914+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:52.234 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:52.234+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:52.885 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:52.885+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:53.264 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:53.264+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:53.871 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:53.870+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:54.312 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:54.312+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:54.872 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:54.872+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:55.348 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:55.348+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:55.902 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:55.901+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:56.304 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:56.303+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:56.628 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:29:56.929 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:56.929+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:57.013 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:29:57.016 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:29:57.027 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":6261,"stamp":"2023-10-06T00:29:56.239131+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001885"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":2083,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:29:39.273204+0000","last_change":"2023-10-06T00:29:39.273204+0000","last_active":"2023-10-06T00:29:39.273204+0000","last_peered":"2023-10-06T00:29:39.273204+0000","last_clean":"2023-10-06T00:29:39.273204+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:29:39.273204+0000","last_undegraded":"2023-10-06T00:29:39.273204+0000","last_fullsized":"2023-10-06T00:29:39.273204+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1986,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:29:32.249345+0000","last_change":"2023-10-06T00:29:32.249345+0000","last_active":"2023-10-06T00:29:32.249345+0000","last_peered":"2023-10-06T00:29:32.249345+0000","last_clean":"2023-10-06T00:29:32.249345+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:29:32.249345+0000","last_undegraded":"2023-10-06T00:29:32.249345+0000","last_fullsized":"2023-10-06T00:29:32.249345+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8622,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:26:47.872288+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:26:47.872288+0000","last_peered":"2023-10-06T00:26:47.872288+0000","last_clean":"2023-10-06T00:26:47.872288+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:26:47.872288+0000","last_undegraded":"2023-10-06T00:26:47.872288+0000","last_fullsized":"2023-10-06T00:26:47.872288+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 773s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11238,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:29:43.241544+0000","last_change":"2023-10-06T00:29:43.241544+0000","last_active":"2023-10-06T00:29:43.241544+0000","last_peered":"2023-10-06T00:29:43.241544+0000","last_clean":"2023-10-06T00:29:43.241544+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:29:43.241544+0000","last_undegraded":"2023-10-06T00:29:43.241544+0000","last_fullsized":"2023-10-06T00:29:43.241544+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2502,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:29:37.281397+0000","last_change":"2023-10-06T00:29:37.281397+0000","last_active":"2023-10-06T00:29:37.281397+0000","last_peered":"2023-10-06T00:29:37.281397+0000","last_clean":"2023-10-06T00:29:37.281397+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:29:37.281397+0000","last_undegraded":"2023-10-06T00:29:37.281397+0000","last_fullsized":"2023-10-06T00:29:37.281397+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11886,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:29:42.241267+0000","last_change":"2023-10-06T00:29:42.241267+0000","last_active":"2023-10-06T00:29:42.241267+0000","last_peered":"2023-10-06T00:29:42.241267+0000","last_clean":"2023-10-06T00:29:42.241267+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:29:42.241267+0000","last_undegraded":"2023-10-06T00:29:42.241267+0000","last_fullsized":"2023-10-06T00:29:42.241267+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11618,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:26:49.384875+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-06T00:26:49.384875+0000","last_peered":"2023-10-06T00:26:49.384875+0000","last_clean":"2023-10-06T00:26:49.384875+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-06T00:26:49.384875+0000","last_undegraded":"2023-10-06T00:26:49.384875+0000","last_fullsized":"2023-10-06T00:26:49.384875+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1788,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:26:49.777242+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-06T00:26:49.777242+0000","last_peered":"2023-10-06T00:26:49.777242+0000","last_clean":"2023-10-06T00:26:49.777242+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-06T00:26:49.777242+0000","last_undegraded":"2023-10-06T00:26:49.777242+0000","last_fullsized":"2023-10-06T00:26:49.777242+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1907,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:29:31.278254+0000","last_change":"2023-10-06T00:29:31.278254+0000","last_active":"2023-10-06T00:29:31.278254+0000","last_peered":"2023-10-06T00:29:31.278254+0000","last_clean":"2023-10-06T00:29:31.278254+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:29:31.278254+0000","last_undegraded":"2023-10-06T00:29:31.278254+0000","last_fullsized":"2023-10-06T00:29:31.278254+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740820,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:29:33 2023","interfaces":[{"interface":"back","average":{"1min":0.23699999999999999,"5min":0.23799999999999999,"15min":0.22500000000000001},"min":{"1min":0.16300000000000001,"5min":0.159,"15min":0.13800000000000001},"max":{"1min":0.44900000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.17999999999999999},{"interface":"front","average":{"1min":0.25700000000000001,"5min":0.249,"15min":0.24399999999999999},"min":{"1min":0.186,"5min":0.18099999999999999,"15min":0.17299999999999999},"max":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":1.1659999999999999},"last":0.20899999999999999}]},{"osd":1,"last update":"Fri Oct 6 00:29:29 2023","interfaces":[{"interface":"back","average":{"1min":0.27400000000000002,"5min":0.27900000000000003,"15min":0.26500000000000001},"min":{"1min":0.19900000000000001,"5min":0.186,"15min":0.157},"max":{"1min":0.44600000000000001,"5min":1.198,"15min":1.198},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.28499999999999998,"5min":0.28100000000000003,"15min":0.28000000000000003},"min":{"1min":0.217,"5min":0.19,"15min":0.17999999999999999},"max":{"1min":0.46000000000000002,"5min":0.48199999999999998,"15min":1.2110000000000001},"last":0.253}]}]},{"osd":0,"up_from":8,"seq":34359740820,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.215,"5min":0.20499999999999999,"15min":0.20100000000000001},"min":{"1min":0.14199999999999999,"5min":0.13600000000000001,"15min":0.13600000000000001},"max":{"1min":0.40500000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.187},{"interface":"front","average":{"1min":0.26100000000000001,"5min":0.26500000000000001,"15min":0.26700000000000002},"min":{"1min":0.20000000000000001,"5min":0.16500000000000001,"15min":0.16500000000000001},"max":{"1min":0.44700000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.27600000000000002}]},{"osd":2,"last update":"Fri Oct 6 00:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.245,"5min":0.24299999999999999,"15min":0.23899999999999999},"min":{"1min":0.16800000000000001,"5min":0.16800000000000001,"15min":0.16600000000000001},"max":{"1min":0.39000000000000001,"5min":0.5,"15min":0.5},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29799999999999999},"min":{"1min":0.22900000000000001,"5min":0.217,"15min":0.21299999999999999},"max":{"1min":0.442,"5min":0.53800000000000003,"15min":0.53800000000000003},"last":0.26200000000000001}]}]},{"osd":1,"up_from":8,"seq":34359740815,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.215,"5min":0.20999999999999999,"15min":0.215},"min":{"1min":0.14799999999999999,"5min":0.14000000000000001,"15min":0.13100000000000001},"max":{"1min":0.36299999999999999,"5min":0.45800000000000002,"15min":0.497},"last":0.17599999999999999},{"interface":"front","average":{"1min":0.23400000000000001,"5min":0.22700000000000001,"15min":0.24099999999999999},"min":{"1min":0.16300000000000001,"5min":0.16300000000000001,"15min":0.154},"max":{"1min":0.39800000000000002,"5min":0.432,"15min":0.51100000000000001},"last":0.191}]},{"osd":2,"last update":"Fri Oct 6 00:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.247,"5min":0.23999999999999999,"15min":0.23999999999999999},"min":{"1min":0.182,"5min":0.16800000000000001,"15min":0.16800000000000001},"max":{"1min":0.41699999999999998,"5min":0.45100000000000001,"15min":0.46999999999999997},"last":0.218},{"interface":"front","average":{"1min":0.25900000000000001,"5min":0.254,"15min":0.25700000000000001},"min":{"1min":0.20499999999999999,"5min":0.19700000000000001,"15min":0.17199999999999999},"max":{"1min":0.436,"5min":0.436,"15min":0.54700000000000004},"last":0.20899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:29:57.029 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:29:57.029 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:29:57.029 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:29:57.030 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:29:57.030 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:29:57.030 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:29:57.030 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:29:57.030 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:29:57.031 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:29:57.031 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:29:57.308 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:57.308+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:57.928 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:57.927+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:58.270 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:58.270+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:58.930 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:58.929+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:59.279 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:29:59.278+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:29:59.966 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:29:59.965+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:00.252 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:00.250+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:00.976 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:30:00.975+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:01.269 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:01.268+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:01.941 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:30:01.940+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:02.284 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:02.284+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:02.905 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:30:02.903+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:03.243 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:03.242+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:03.918 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:30:03.916+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:04.237 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:04.235+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:04.936 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:30:04.933+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:05.195 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:05.194+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:05.904 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:30:05.903+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:06.178 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:06.176+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:06.891 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:30:06.890+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:07.144 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:07.143+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:07.902 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:30:07.901+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:08.184 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:08.182+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:08.946 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:30:08.945+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:09.186 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:09.184+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:09.926 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:30:09.925+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:10.184 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:10.182+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:10.942 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:30:10.941+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:11.183 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:11.182+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:11.915 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:30:11.914+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:12.138 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:12.137+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:12.940 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:30:12.939+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:13.128 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:13.126+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:13.955 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:30:13.953+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:14.102 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:14.101+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:14.906 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:30:14.905+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:15.130 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:15.129+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:15.930 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:30:15.929+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:16.093 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:16.091+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:16.949 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:30:16.947+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:17.031 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:30:17.058 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:17.057+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:17.376 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:30:17.379 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:30:17.390 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":6271,"stamp":"2023-10-06T00:30:16.242386+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002011"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":2086,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:30:05.195272+0000","last_change":"2023-10-06T00:30:05.195272+0000","last_active":"2023-10-06T00:30:05.195272+0000","last_peered":"2023-10-06T00:30:05.195272+0000","last_clean":"2023-10-06T00:30:05.195272+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:30:05.195272+0000","last_undegraded":"2023-10-06T00:30:05.195272+0000","last_fullsized":"2023-10-06T00:30:05.195272+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1989,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:29:58.269833+0000","last_change":"2023-10-06T00:29:58.269833+0000","last_active":"2023-10-06T00:29:58.269833+0000","last_peered":"2023-10-06T00:29:58.269833+0000","last_clean":"2023-10-06T00:29:58.269833+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:29:58.269833+0000","last_undegraded":"2023-10-06T00:29:58.269833+0000","last_fullsized":"2023-10-06T00:29:58.269833+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8622,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:26:47.872288+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:26:47.872288+0000","last_peered":"2023-10-06T00:26:47.872288+0000","last_clean":"2023-10-06T00:26:47.872288+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:26:47.872288+0000","last_undegraded":"2023-10-06T00:26:47.872288+0000","last_fullsized":"2023-10-06T00:26:47.872288+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 773s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11241,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:30:10.183781+0000","last_change":"2023-10-06T00:30:10.183781+0000","last_active":"2023-10-06T00:30:10.183781+0000","last_peered":"2023-10-06T00:30:10.183781+0000","last_clean":"2023-10-06T00:30:10.183781+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:30:10.183781+0000","last_undegraded":"2023-10-06T00:30:10.183781+0000","last_fullsized":"2023-10-06T00:30:10.183781+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2505,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:30:03.243298+0000","last_change":"2023-10-06T00:30:03.243298+0000","last_active":"2023-10-06T00:30:03.243298+0000","last_peered":"2023-10-06T00:30:03.243298+0000","last_clean":"2023-10-06T00:30:03.243298+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:30:03.243298+0000","last_undegraded":"2023-10-06T00:30:03.243298+0000","last_fullsized":"2023-10-06T00:30:03.243298+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11889,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:30:06.177773+0000","last_change":"2023-10-06T00:30:06.177773+0000","last_active":"2023-10-06T00:30:06.177773+0000","last_peered":"2023-10-06T00:30:06.177773+0000","last_clean":"2023-10-06T00:30:06.177773+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:30:06.177773+0000","last_undegraded":"2023-10-06T00:30:06.177773+0000","last_fullsized":"2023-10-06T00:30:06.177773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11618,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:26:49.384875+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-06T00:26:49.384875+0000","last_peered":"2023-10-06T00:26:49.384875+0000","last_clean":"2023-10-06T00:26:49.384875+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-06T00:26:49.384875+0000","last_undegraded":"2023-10-06T00:26:49.384875+0000","last_fullsized":"2023-10-06T00:26:49.384875+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1788,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:26:49.777242+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-06T00:26:49.777242+0000","last_peered":"2023-10-06T00:26:49.777242+0000","last_clean":"2023-10-06T00:26:49.777242+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-06T00:26:49.777242+0000","last_undegraded":"2023-10-06T00:26:49.777242+0000","last_fullsized":"2023-10-06T00:26:49.777242+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1910,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:29:56.303560+0000","last_change":"2023-10-06T00:29:56.303560+0000","last_active":"2023-10-06T00:29:56.303560+0000","last_peered":"2023-10-06T00:29:56.303560+0000","last_clean":"2023-10-06T00:29:56.303560+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:29:56.303560+0000","last_undegraded":"2023-10-06T00:29:56.303560+0000","last_fullsized":"2023-10-06T00:29:56.303560+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740824,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:29:33 2023","interfaces":[{"interface":"back","average":{"1min":0.23699999999999999,"5min":0.23799999999999999,"15min":0.22500000000000001},"min":{"1min":0.16300000000000001,"5min":0.159,"15min":0.13800000000000001},"max":{"1min":0.44900000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.14499999999999999},{"interface":"front","average":{"1min":0.25700000000000001,"5min":0.249,"15min":0.24399999999999999},"min":{"1min":0.186,"5min":0.18099999999999999,"15min":0.17299999999999999},"max":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":1.1659999999999999},"last":0.17699999999999999}]},{"osd":1,"last update":"Fri Oct 6 00:29:29 2023","interfaces":[{"interface":"back","average":{"1min":0.27400000000000002,"5min":0.27900000000000003,"15min":0.26500000000000001},"min":{"1min":0.19900000000000001,"5min":0.186,"15min":0.157},"max":{"1min":0.44600000000000001,"5min":1.198,"15min":1.198},"last":0.192},{"interface":"front","average":{"1min":0.28499999999999998,"5min":0.28100000000000003,"15min":0.28000000000000003},"min":{"1min":0.217,"5min":0.19,"15min":0.17999999999999999},"max":{"1min":0.46000000000000002,"5min":0.48199999999999998,"15min":1.2110000000000001},"last":0.21099999999999999}]}]},{"osd":0,"up_from":8,"seq":34359740824,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.215,"5min":0.20499999999999999,"15min":0.20100000000000001},"min":{"1min":0.14199999999999999,"5min":0.13600000000000001,"15min":0.13600000000000001},"max":{"1min":0.40500000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.26100000000000001,"5min":0.26500000000000001,"15min":0.26700000000000002},"min":{"1min":0.20000000000000001,"5min":0.16500000000000001,"15min":0.16500000000000001},"max":{"1min":0.44700000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.24399999999999999}]},{"osd":2,"last update":"Fri Oct 6 00:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.245,"5min":0.24299999999999999,"15min":0.23899999999999999},"min":{"1min":0.16800000000000001,"5min":0.16800000000000001,"15min":0.16600000000000001},"max":{"1min":0.39000000000000001,"5min":0.5,"15min":0.5},"last":0.20300000000000001},{"interface":"front","average":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29799999999999999},"min":{"1min":0.22900000000000001,"5min":0.217,"15min":0.21299999999999999},"max":{"1min":0.442,"5min":0.53800000000000003,"15min":0.53800000000000003},"last":0.29399999999999998}]}]},{"osd":1,"up_from":8,"seq":34359740819,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:30:01 2023","interfaces":[{"interface":"back","average":{"1min":0.19800000000000001,"5min":0.20699999999999999,"15min":0.21299999999999999},"min":{"1min":0.154,"5min":0.14000000000000001,"15min":0.13100000000000001},"max":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"last":0.20300000000000001},{"interface":"front","average":{"1min":0.221,"5min":0.22500000000000001,"15min":0.23799999999999999},"min":{"1min":0.182,"5min":0.16300000000000001,"15min":0.16300000000000001},"max":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.47599999999999998},"last":0.23999999999999999}]},{"osd":2,"last update":"Fri Oct 6 00:30:01 2023","interfaces":[{"interface":"back","average":{"1min":0.22700000000000001,"5min":0.23699999999999999,"15min":0.23999999999999999},"min":{"1min":0.182,"5min":0.17899999999999999,"15min":0.16800000000000001},"max":{"1min":0.433,"5min":0.45100000000000001,"15min":0.45100000000000001},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.24299999999999999,"5min":0.252,"15min":0.254},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.17199999999999999},"max":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.50900000000000001},"last":0.253}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:30:17.392 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:30:17.392 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:30:17.392 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:30:17.393 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:30:17.393 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:30:17.393 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:30:17.393 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:30:17.393 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:30:17.394 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:30:17.394 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T00:30:17.958 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:30:17.957+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:18.066 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:18.065+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:18.932 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:30:18.931+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:19.101 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:19.099+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:19.944 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:30:19.942+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:20.054 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:20.053+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:20.983 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:30:20.981+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:21.031 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:21.030+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:21.962 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:30:21.961+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:22.026 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:22.025+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:23.000 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:30:22.998+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:23.000 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:22.999+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:23.997 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:23.996+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:24.013 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:30:24.011+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:25.012 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:25.010+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:25.046 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:30:25.044+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:26.012 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:26.011+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:26.057 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:30:26.055+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:26.989 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:26.987+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:27.024 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:30:27.022+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:27.946 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:27.944+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:28.018 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:30:28.017+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:28.928 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:28.927+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:28.982 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:30:28.980+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:29.917 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:29.916+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:29.990 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:30:29.989+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:30.888 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:30.887+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:31.019 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:30:31.017+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:31.852 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:31.850+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:31.982 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:30:31.980+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:32.813 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:32.811+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:33.010 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:30:33.008+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:33.790 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:33.788+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:33.985 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:30:33.984+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:34.756 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:34.755+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:34.950 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:30:34.948+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:35.764 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:35.762+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:35.988 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:30:35.986+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:36.767 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:36.765+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:37.038 INFO:tasks.ceph.osd.0.smithi167.stderr:2023-10-06T00:30:37.037+0000 7ff685387700 -1 osd.0 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:344 3.3 3:cc24c031:test1::rbd_data.29569a84ce07.000000000000006a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2768896~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:37.395 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T00:30:37.753 INFO:tasks.ceph.osd.1.smithi167.stderr:2023-10-06T00:30:37.751+0000 7f6cbabe8700 -1 osd.1 160 get_health_metrics reporting 256 slow ops, oldest is osd_op(client.10585.0:319 3.2 3:66873d86:test1::rbd_data.29569a84ce07.00000000000000fc:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1581056~4096 [fadvise_random] in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158) 2023-10-06T00:30:37.755 INFO:teuthology.orchestra.run.smithi167.stdout: 2023-10-06T00:30:37.758 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2023-10-06T00:30:37.769 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":6281,"stamp":"2023-10-06T00:30:36.245631+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209360996,"num_objects":318,"num_object_clones":1,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":178,"num_read":65755,"num_read_kb":127361,"num_write":17557,"num_write_kb":8828012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25008,"ondisk_log_size":25008,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":929508,"kb_used_data":25356,"kb_used_omap":0,"kb_used_meta":904128,"kb_avail":282186012,"statfs":{"total":289910292480,"available":288958476288,"internally_reserved":0,"allocated":25964544,"data_stored":324876177,"data_compressed":687682,"data_compressed_allocated":20324352,"data_compressed_original":321617920,"omap_allocated":0,"internal_metadata":925827072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,512],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001861"},"pg_stats":[{"pgid":"2.7","version":"158'862","reported_seq":2086,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:30:05.195272+0000","last_change":"2023-10-06T00:30:05.195272+0000","last_active":"2023-10-06T00:30:05.195272+0000","last_peered":"2023-10-06T00:30:05.195272+0000","last_clean":"2023-10-06T00:30:05.195272+0000","last_became_active":"2023-10-05T21:06:33.648599+0000","last_became_peered":"2023-10-05T21:06:33.648599+0000","last_unstale":"2023-10-06T00:30:05.195272+0000","last_undegraded":"2023-10-06T00:30:05.195272+0000","last_fullsized":"2023-10-06T00:30:05.195272+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017824889999999999,"stat_sum":{"num_bytes":15896777,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":911,"num_read_kb":459,"num_write":715,"num_write_kb":1142088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"158'888","reported_seq":1992,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:30:30.888083+0000","last_change":"2023-10-06T00:30:30.888083+0000","last_active":"2023-10-06T00:30:30.888083+0000","last_peered":"2023-10-06T00:30:30.888083+0000","last_clean":"2023-10-06T00:30:30.888083+0000","last_became_active":"2023-10-05T21:06:33.648408+0000","last_became_peered":"2023-10-05T21:06:33.648408+0000","last_unstale":"2023-10-06T00:30:30.888083+0000","last_undegraded":"2023-10-06T00:30:30.888083+0000","last_fullsized":"2023-10-06T00:30:30.888083+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001719943,"stat_sum":{"num_bytes":19105362,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":810,"num_read_kb":3424,"num_write":699,"num_write_kb":1113181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"160'790","reported_seq":1670,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.055869+0000","last_change":"2023-10-06T00:15:59.055836+0000","last_active":"2023-10-06T00:15:59.055869+0000","last_peered":"2023-10-06T00:15:59.055869+0000","last_clean":"2023-10-06T00:15:59.055869+0000","last_became_active":"2023-10-05T21:06:33.650979+0000","last_became_peered":"2023-10-05T21:06:33.650979+0000","last_unstale":"2023-10-06T00:15:59.055869+0000","last_undegraded":"2023-10-06T00:15:59.055869+0000","last_fullsized":"2023-10-06T00:15:59.055869+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'790","last_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_deep_scrub":"160'790","last_deep_scrub_stamp":"2023-10-06T00:15:59.055718+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.055718+0000","objects_scrubbed":15,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:27.410311+0000","scrub_duration":0.090026611000000006,"objects_trimmed":0,"snaptrim_duration":0.001787351,"stat_sum":{"num_bytes":32161942,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":570,"num_read_kb":4841,"num_write":633,"num_write_kb":1044320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":222,"num_omap_keys":25,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"160'3241","reported_seq":8622,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-06T00:26:47.872288+0000","last_change":"2023-10-06T00:16:00.294886+0000","last_active":"2023-10-06T00:26:47.872288+0000","last_peered":"2023-10-06T00:26:47.872288+0000","last_clean":"2023-10-06T00:26:47.872288+0000","last_became_active":"2023-10-05T21:06:40.688343+0000","last_became_peered":"2023-10-05T21:06:40.688343+0000","last_unstale":"2023-10-06T00:26:47.872288+0000","last_undegraded":"2023-10-06T00:26:47.872288+0000","last_fullsized":"2023-10-06T00:26:47.872288+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":24,"log_size":3241,"log_dups_size":0,"ondisk_log_size":3241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 773s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263634,"stat_sum":{"num_bytes":7143762,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":44,"num_read":12160,"num_read_kb":24110,"num_write":1945,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":338,"num_flush":182,"num_flush_kb":1091,"num_evict":505,"num_evict_kb":25304,"num_promote":554,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'1295","reported_seq":2186,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:35.205127+0000","last_change":"2023-10-06T00:13:35.205095+0000","last_active":"2023-10-06T00:13:35.205127+0000","last_peered":"2023-10-06T00:13:35.205127+0000","last_clean":"2023-10-06T00:13:35.205127+0000","last_became_active":"2023-10-05T21:06:33.650750+0000","last_became_peered":"2023-10-05T21:06:33.650750+0000","last_unstale":"2023-10-06T00:13:35.205127+0000","last_undegraded":"2023-10-06T00:13:35.205127+0000","last_fullsized":"2023-10-06T00:13:35.205127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1295","last_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_deep_scrub":"160'1295","last_deep_scrub_stamp":"2023-10-06T00:13:35.204978+0000","last_clean_scrub_stamp":"2023-10-06T00:13:35.204978+0000","objects_scrubbed":13,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:23.007972+0000","scrub_duration":0.054871009999999998,"objects_trimmed":0,"snaptrim_duration":0.00035888800000000002,"stat_sum":{"num_bytes":25379017,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":597,"num_read_kb":3698,"num_write":714,"num_write_kb":1095897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"158'4809","reported_seq":11241,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:30:10.183781+0000","last_change":"2023-10-06T00:30:10.183781+0000","last_active":"2023-10-06T00:30:10.183781+0000","last_peered":"2023-10-06T00:30:10.183781+0000","last_clean":"2023-10-06T00:30:10.183781+0000","last_became_active":"2023-10-05T21:06:40.688198+0000","last_became_peered":"2023-10-05T21:06:40.688198+0000","last_unstale":"2023-10-06T00:30:10.183781+0000","last_undegraded":"2023-10-06T00:30:10.183781+0000","last_fullsized":"2023-10-06T00:30:10.183781+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115835,"stat_sum":{"num_bytes":3842921,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":46,"num_read":17971,"num_read_kb":19144,"num_write":3973,"num_write_kb":2528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":857,"num_flush":173,"num_flush_kb":1088,"num_evict":590,"num_evict_kb":31153,"num_promote":652,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"160'99","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:15:59.997723+0000","last_change":"2023-10-06T00:15:59.997699+0000","last_active":"2023-10-06T00:15:59.997723+0000","last_peered":"2023-10-06T00:15:59.997723+0000","last_clean":"2023-10-06T00:15:59.997723+0000","last_became_active":"2023-10-05T21:06:29.921153+0000","last_became_peered":"2023-10-05T21:06:29.921153+0000","last_unstale":"2023-10-06T00:15:59.997723+0000","last_undegraded":"2023-10-06T00:15:59.997723+0000","last_fullsized":"2023-10-06T00:15:59.997723+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'99","last_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_deep_scrub":"160'99","last_deep_scrub_stamp":"2023-10-06T00:15:59.997619+0000","last_clean_scrub_stamp":"2023-10-06T00:15:59.997619+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:55.685653+0000","scrub_duration":0.0068997570000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"158'1306","reported_seq":2505,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:30:03.243298+0000","last_change":"2023-10-06T00:30:03.243298+0000","last_active":"2023-10-06T00:30:03.243298+0000","last_peered":"2023-10-06T00:30:03.243298+0000","last_clean":"2023-10-06T00:30:03.243298+0000","last_became_active":"2023-10-05T21:06:33.649481+0000","last_became_peered":"2023-10-05T21:06:33.649481+0000","last_unstale":"2023-10-06T00:30:03.243298+0000","last_undegraded":"2023-10-06T00:30:03.243298+0000","last_fullsized":"2023-10-06T00:30:03.243298+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055919400000000005,"stat_sum":{"num_bytes":24779522,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":868,"num_read_kb":6103,"num_write":743,"num_write_kb":1158723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'5111","reported_seq":11889,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:30:06.177773+0000","last_change":"2023-10-06T00:30:06.177773+0000","last_active":"2023-10-06T00:30:06.177773+0000","last_peered":"2023-10-06T00:30:06.177773+0000","last_clean":"2023-10-06T00:30:06.177773+0000","last_became_active":"2023-10-05T21:06:40.688262+0000","last_became_peered":"2023-10-05T21:06:40.688262+0000","last_unstale":"2023-10-06T00:30:06.177773+0000","last_undegraded":"2023-10-06T00:30:06.177773+0000","last_fullsized":"2023-10-06T00:30:06.177773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":5111,"log_dups_size":0,"ondisk_log_size":5111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637299,"stat_sum":{"num_bytes":15803693,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":43,"num_read":15680,"num_read_kb":16601,"num_write":3260,"num_write_kb":1534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":5,"num_bytes_hit_set_archive":1227,"num_flush":222,"num_flush_kb":3940,"num_evict":467,"num_evict_kb":20307,"num_promote":522,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"160'895","reported_seq":1873,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:13:41.139002+0000","last_change":"2023-10-06T00:13:41.138979+0000","last_active":"2023-10-06T00:13:41.139002+0000","last_peered":"2023-10-06T00:13:41.139002+0000","last_clean":"2023-10-06T00:13:41.139002+0000","last_became_active":"2023-10-05T21:06:33.650867+0000","last_became_peered":"2023-10-05T21:06:33.650867+0000","last_unstale":"2023-10-06T00:13:41.139002+0000","last_undegraded":"2023-10-06T00:13:41.139002+0000","last_fullsized":"2023-10-06T00:13:41.139002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'895","last_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_deep_scrub":"160'895","last_deep_scrub_stamp":"2023-10-06T00:13:41.138900+0000","last_clean_scrub_stamp":"2023-10-06T00:13:41.138900+0000","objects_scrubbed":11,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:13.064510+0000","scrub_duration":0.028422307000000001,"objects_trimmed":0,"snaptrim_duration":0.0013498169999999999,"stat_sum":{"num_bytes":7385308,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":789,"num_read_kb":531,"num_write":734,"num_write_kb":1168889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":295,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"158'4056","reported_seq":11618,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:26:49.384875+0000","last_change":"2023-10-05T21:11:08.651877+0000","last_active":"2023-10-06T00:26:49.384875+0000","last_peered":"2023-10-06T00:26:49.384875+0000","last_clean":"2023-10-06T00:26:49.384875+0000","last_became_active":"2023-10-05T21:06:40.688203+0000","last_became_peered":"2023-10-05T21:06:40.688203+0000","last_unstale":"2023-10-06T00:26:49.384875+0000","last_undegraded":"2023-10-06T00:26:49.384875+0000","last_fullsized":"2023-10-06T00:26:49.384875+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:39.652014+0000","last_clean_scrub_stamp":"2023-10-05T21:06:39.652014+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012094790000000001,"stat_sum":{"num_bytes":7152334,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":44,"num_read":13882,"num_read_kb":42286,"num_write":2652,"num_write_kb":1602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":718,"num_flush":199,"num_flush_kb":3239,"num_evict":466,"num_evict_kb":42616,"num_promote":523,"num_flush_mode_high":0,"num_flush_mode_low":1,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"158'804","reported_seq":1788,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:26:49.777242+0000","last_change":"2023-10-05T21:11:08.649905+0000","last_active":"2023-10-06T00:26:49.777242+0000","last_peered":"2023-10-06T00:26:49.777242+0000","last_clean":"2023-10-06T00:26:49.777242+0000","last_became_active":"2023-10-05T21:06:33.648745+0000","last_became_peered":"2023-10-05T21:06:33.648745+0000","last_unstale":"2023-10-06T00:26:49.777242+0000","last_undegraded":"2023-10-06T00:26:49.777242+0000","last_fullsized":"2023-10-06T00:26:49.777242+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041327900000000001,"stat_sum":{"num_bytes":24954010,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":750,"num_read_kb":5047,"num_write":663,"num_write_kb":1051328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"158'852","reported_seq":1913,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-06T00:30:26.988672+0000","last_change":"2023-10-06T00:30:26.988672+0000","last_active":"2023-10-06T00:30:26.988672+0000","last_peered":"2023-10-06T00:30:26.988672+0000","last_clean":"2023-10-06T00:30:26.988672+0000","last_became_active":"2023-10-05T21:06:33.651058+0000","last_became_peered":"2023-10-05T21:06:33.651058+0000","last_unstale":"2023-10-06T00:30:26.988672+0000","last_undegraded":"2023-10-06T00:30:26.988672+0000","last_fullsized":"2023-10-06T00:30:26.988672+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T21:06:32.625712+0000","last_clean_scrub_stamp":"2023-10-05T21:06:32.625712+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015930569999999999,"stat_sum":{"num_bytes":25165980,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":661,"num_read_kb":1026,"num_write":689,"num_write_kb":1045186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":33942710,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":177,"num_read":59693,"num_read_kb":102141,"num_write":11830,"num_write_kb":6506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":13,"num_bytes_hit_set_archive":3140,"num_flush":776,"num_flush_kb":9358,"num_evict":2028,"num_evict_kb":119380,"num_promote":2251,"num_flush_mode_high":0,"num_flush_mode_low":2,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":48568684,"data_compressed":70498,"data_compressed_allocated":3039232,"data_compressed_original":47906816,"omap_allocated":0,"internal_metadata":0},"log_size":17217,"ondisk_log_size":17217,"up":8,"acting":8,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":6810,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":174827918,"num_objects":109,"num_object_clones":1,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":1,"num_read":5956,"num_read_kb":25129,"num_write":5590,"num_write_kb":8819612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":518,"num_omap_keys":58,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18014208,"data_stored":273060636,"data_compressed":610374,"data_compressed_allocated":17211392,"data_compressed_original":272564224,"omap_allocated":0,"internal_metadata":0},"log_size":7692,"ondisk_log_size":7692,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359740828,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302912,"kb_used_data":5624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94068928,"statfs":{"total":96636764160,"available":96326582272,"internally_reserved":0,"allocated":5758976,"data_stored":60734368,"data_compressed":194516,"data_compressed_allocated":3833856,"data_compressed_original":59572224,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:29:33 2023","interfaces":[{"interface":"back","average":{"1min":0.23699999999999999,"5min":0.23799999999999999,"15min":0.22500000000000001},"min":{"1min":0.16300000000000001,"5min":0.159,"15min":0.13800000000000001},"max":{"1min":0.44900000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.16700000000000001},{"interface":"front","average":{"1min":0.25700000000000001,"5min":0.249,"15min":0.24399999999999999},"min":{"1min":0.186,"5min":0.18099999999999999,"15min":0.17299999999999999},"max":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":1.1659999999999999},"last":0.20999999999999999}]},{"osd":1,"last update":"Fri Oct 6 00:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.25900000000000001,"5min":0.27300000000000002,"15min":0.26500000000000001},"min":{"1min":0.192,"5min":0.186,"15min":0.157},"max":{"1min":0.377,"5min":1.198,"15min":1.198},"last":0.19800000000000001},{"interface":"front","average":{"1min":0.27700000000000002,"5min":0.28199999999999997,"15min":0.28100000000000003},"min":{"1min":0.21099999999999999,"5min":0.19,"15min":0.17999999999999999},"max":{"1min":0.39400000000000002,"5min":0.48199999999999998,"15min":1.2110000000000001},"last":0.24299999999999999}]}]},{"osd":0,"up_from":8,"seq":34359740828,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318064,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94053776,"statfs":{"total":96636764160,"available":96311066624,"internally_reserved":0,"allocated":8691712,"data_stored":112358762,"data_compressed":164331,"data_compressed_allocated":6975488,"data_compressed_original":111411200,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 00:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.218,"5min":0.20699999999999999,"15min":0.20300000000000001},"min":{"1min":0.16200000000000001,"5min":0.13700000000000001,"15min":0.13600000000000001},"max":{"1min":0.30499999999999999,"5min":0.41999999999999998,"15min":0.49399999999999999},"last":0.248},{"interface":"front","average":{"1min":0.25900000000000001,"5min":0.26000000000000001,"15min":0.26600000000000001},"min":{"1min":0.218,"5min":0.17399999999999999,"15min":0.16500000000000001},"max":{"1min":0.35099999999999998,"5min":0.44800000000000001,"15min":0.45300000000000001},"last":0.26200000000000001}]},{"osd":2,"last update":"Fri Oct 6 00:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.23499999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"min":{"1min":0.193,"5min":0.16800000000000001,"15min":0.16600000000000001},"max":{"1min":0.32000000000000001,"5min":0.41699999999999998,"15min":0.5},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.29099999999999998,"5min":0.29399999999999998,"15min":0.29899999999999999},"min":{"1min":0.24199999999999999,"5min":0.217,"15min":0.215},"max":{"1min":0.34999999999999998,"5min":0.47699999999999998,"15min":0.53800000000000003},"last":0.31}]}]},{"osd":1,"up_from":8,"seq":34359740823,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308532,"kb_used_data":11244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94063308,"statfs":{"total":96636764160,"available":96320827392,"internally_reserved":0,"allocated":11513856,"data_stored":151783047,"data_compressed":328835,"data_compressed_allocated":9515008,"data_compressed_original":150634496,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256],"upper_bound":33554432},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 00:30:01 2023","interfaces":[{"interface":"back","average":{"1min":0.19800000000000001,"5min":0.20699999999999999,"15min":0.21299999999999999},"min":{"1min":0.154,"5min":0.14000000000000001,"15min":0.13100000000000001},"max":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"last":0.19},{"interface":"front","average":{"1min":0.221,"5min":0.22500000000000001,"15min":0.23799999999999999},"min":{"1min":0.182,"5min":0.16300000000000001,"15min":0.16300000000000001},"max":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.47599999999999998},"last":0.22500000000000001}]},{"osd":2,"last update":"Fri Oct 6 00:30:01 2023","interfaces":[{"interface":"back","average":{"1min":0.22700000000000001,"5min":0.23699999999999999,"15min":0.23999999999999999},"min":{"1min":0.182,"5min":0.17899999999999999,"15min":0.16800000000000001},"max":{"1min":0.433,"5min":0.45100000000000001,"15min":0.45100000000000001},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.24299999999999999,"5min":0.252,"15min":0.254},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.17199999999999999},"max":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.50900000000000001},"last":0.26500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":3405,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":100768999,"data_compressed":145920,"data_compressed_allocated":6291456,"data_compressed_original":100663296,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9007104,"data_stored":136530318,"data_compressed":305187,"data_compressed_allocated":8605696,"data_compressed_original":136282112,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2510848,"data_stored":35761319,"data_compressed":159267,"data_compressed_allocated":2314240,"data_compressed_original":35618816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":10310688,"data_compressed":15006,"data_compressed_allocated":647168,"data_compressed_original":10174464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1110016,"data_stored":13973654,"data_compressed":20243,"data_compressed_allocated":872448,"data_compressed_original":13778944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1908736,"data_stored":24284342,"data_compressed":35249,"data_compressed_allocated":1519616,"data_compressed_original":23953408,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T00:30:37.771 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:30:37.771 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:30:37.771 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:30:37.772 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:30:37.772 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:30:37.772 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:30:37.773 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-05T21:06:39.652014+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=39, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:30:37.773 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:30:37.773 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-05T21:06:32.625712+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=5, tm_hour=21, tm_min=6, tm_sec=32, tm_wday=3, tm_yday=278, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=0, tm_min=9, tm_sec=32, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T00:30:37.773 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph-c_487ffcb5ddb60d29c5688d1a457e812b1b697b90/qa/tasks/ceph.py", line 1907, in task osd_scrub_pgs(ctx, config) File "/home/teuthworker/src/github.com_ceph_ceph-c_487ffcb5ddb60d29c5688d1a457e812b1b697b90/qa/tasks/ceph.py", line 1302, in osd_scrub_pgs raise RuntimeError('Exiting scrub checking -- not all pgs scrubbed.') RuntimeError: Exiting scrub checking -- not all pgs scrubbed. 2023-10-06T00:30:37.783 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-06T00:30:37.783 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-06T00:30:37.783 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-06T00:30:37.784 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T00:30:37.817 INFO:tasks.ceph.osd.0:Stopped 2023-10-06T00:30:37.817 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-06T00:30:37.817 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T00:30:37.890 INFO:tasks.ceph.osd.1:Stopped 2023-10-06T00:30:37.890 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-06T00:30:37.891 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T00:30:37.958 INFO:tasks.ceph.osd.2:Stopped 2023-10-06T00:30:37.958 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-06T00:30:37.958 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-06T00:30:37.958 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T00:30:38.022 INFO:tasks.ceph.mgr.x:Stopped 2023-10-06T00:30:38.023 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-06T00:30:38.023 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-06T00:30:38.023 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T00:30:38.080 INFO:tasks.ceph.mon.a:Stopped 2023-10-06T00:30:38.080 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-06T00:30:38.081 DEBUG:teuthology.orchestra.run.smithi167:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | head -n 1 2023-10-06T00:30:38.149 INFO:teuthology.orchestra.run.smithi167.stdout:1696540318.739903 osd.0 (osd.0) 3 : cluster [WRN] 161 slow requests (by type [ 'reached pg' : 43 'delayed' : 118 ] most affected pool [ 'cache' : 161 ]) 2023-10-06T00:30:38.151 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-10-06T00:30:38.151 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T00:30:38.151 DEBUG:teuthology.orchestra.run.smithi167:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-06T00:30:38.711 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T00:30:38.711 DEBUG:teuthology.orchestra.run.smithi167:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-06T00:30:38.821 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T00:30:38.821 DEBUG:teuthology.orchestra.run.smithi167:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-06T00:30:38.898 INFO:tasks.ceph:Archiving mon data... 2023-10-06T00:30:38.899 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-10-05_14:31:59-rbd-wip-yuri2-testing-2023-10-04-1333-quincy-distro-default-smithi/7411791/data/mon.a.tgz 2023-10-06T00:30:38.900 DEBUG:teuthology.orchestra.run.smithi167:> mktemp 2023-10-06T00:30:38.919 INFO:teuthology.orchestra.run.smithi167.stdout:/tmp/tmp.X0gKlmQAFm 2023-10-06T00:30:38.919 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.X0gKlmQAFm 2023-10-06T00:30:39.064 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 0666 /tmp/tmp.X0gKlmQAFm 2023-10-06T00:30:39.152 DEBUG:teuthology.orchestra.remote:smithi167:/tmp/tmp.X0gKlmQAFm is 449KB 2023-10-06T00:30:39.250 DEBUG:teuthology.orchestra.run.smithi167:> rm -fr /tmp/tmp.X0gKlmQAFm 2023-10-06T00:30:39.268 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-06T00:30:39.268 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-06T00:30:39.334 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-06T00:30:39.334 DEBUG:teuthology.orchestra.run.smithi167:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-06T00:30:39.402 INFO:teuthology.orchestra.run.smithi167.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-06T00:30:39.408 INFO:tasks.ceph:Archiving crash dumps... 2023-10-06T00:30:39.410 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-05_14:31:59-rbd-wip-yuri2-testing-2023-10-04-1333-quincy-distro-default-smithi/7411791/remote/smithi167/crash 2023-10-06T00:30:39.411 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-06T00:30:39.475 INFO:tasks.ceph:Compressing logs... 2023-10-06T00:30:39.476 DEBUG:teuthology.orchestra.run.smithi167:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T00:32:50.724 INFO:tasks.ceph:Archiving logs... 2023-10-06T00:32:50.725 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-05_14:31:59-rbd-wip-yuri2-testing-2023-10-04-1333-quincy-distro-default-smithi/7411791/remote/smithi167/log 2023-10-06T00:32:50.726 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T00:33:01.355 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_487ffcb5ddb60d29c5688d1a457e812b1b697b90/qa/tasks/ceph.py", line 1907, in task osd_scrub_pgs(ctx, config) File "/home/teuthworker/src/github.com_ceph_ceph-c_487ffcb5ddb60d29c5688d1a457e812b1b697b90/qa/tasks/ceph.py", line 1302, in osd_scrub_pgs raise RuntimeError('Exiting scrub checking -- not all pgs scrubbed.') RuntimeError: Exiting scrub checking -- not all pgs scrubbed. 2023-10-06T00:33:01.357 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-06T00:33:01.376 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_487ffcb5ddb60d29c5688d1a457e812b1b697b90/qa/tasks/ceph.py", line 1907, in task osd_scrub_pgs(ctx, config) File "/home/teuthworker/src/github.com_ceph_ceph-c_487ffcb5ddb60d29c5688d1a457e812b1b697b90/qa/tasks/ceph.py", line 1302, in osd_scrub_pgs raise RuntimeError('Exiting scrub checking -- not all pgs scrubbed.') RuntimeError: Exiting scrub checking -- not all pgs scrubbed. 2023-10-06T00:33:01.378 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-06T00:33:01.378 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T00:33:01.440 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-06T00:33:01.449 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-06T00:33:01.449 DEBUG:teuthology.orchestra.run.smithi167:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T00:33:01.473 INFO:teuthology.orchestra.run.smithi167.stderr:bash: ntpq: command not found 2023-10-06T00:33:01.477 INFO:teuthology.orchestra.run.smithi167.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T00:33:01.478 INFO:teuthology.orchestra.run.smithi167.stdout:=============================================================================== 2023-10-06T00:33:01.478 INFO:teuthology.orchestra.run.smithi167.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 68 -1524ns[ +38us] +/- 75ms 2023-10-06T00:33:01.478 INFO:teuthology.orchestra.run.smithi167.stdout:^* hv02.front.sepia.ceph.com 3 9 377 2 -222us[ -180us] +/- 55ms 2023-10-06T00:33:01.478 INFO:teuthology.orchestra.run.smithi167.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 74 +307us[ +346us] +/- 87ms 2023-10-06T00:33:01.478 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T00:33:01.479 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-06T00:33:01.488 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-06T00:33:01.488 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-06T00:33:01.497 DEBUG:teuthology.orchestra.run.smithi167:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-06T00:33:01.617 DEBUG:teuthology.orchestra.run.smithi167:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T00:33:01.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T00:33:01.650 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-06T00:33:01.659 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-06T00:33:01.668 INFO:teuthology.task.internal:Duration was 15379.127360 seconds 2023-10-06T00:33:01.669 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-06T00:33:01.678 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-06T00:33:01.678 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T00:33:01.737 INFO:teuthology.orchestra.run.smithi167.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T00:33:01.840 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-06T00:33:01.840 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi167.front.sepia.ceph.com 2023-10-06T00:33:01.841 DEBUG:teuthology.orchestra.run.smithi167:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T00:33:01.863 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-06T00:33:01.863 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T00:33:01.943 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-06T00:33:01.943 DEBUG:teuthology.orchestra.run.smithi167:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T00:33:02.053 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-06T00:33:02.063 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-06T00:33:02.063 DEBUG:teuthology.orchestra.run.smithi167:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T00:33:02.091 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-06T00:33:02.100 DEBUG:teuthology.orchestra.run.smithi167:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T00:33:02.158 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = core 2023-10-06T00:33:02.185 DEBUG:teuthology.orchestra.run.smithi167:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T00:33:02.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T00:33:02.243 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-06T00:33:02.252 INFO:teuthology.task.internal:Transferring archived files... 2023-10-06T00:33:02.254 DEBUG:teuthology.misc:Transferring archived files from smithi167:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-05_14:31:59-rbd-wip-yuri2-testing-2023-10-04-1333-quincy-distro-default-smithi/7411791/remote/smithi167 2023-10-06T00:33:02.254 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T00:33:02.321 INFO:teuthology.task.internal:Removing archive directory... 2023-10-06T00:33:02.321 DEBUG:teuthology.orchestra.run.smithi167:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T00:33:02.373 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-06T00:33:02.384 INFO:teuthology.task.internal:Not uploading archives. 2023-10-06T00:33:02.384 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-06T00:33:02.394 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-06T00:33:02.394 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T00:33:02.431 INFO:teuthology.orchestra.run.smithi167.stdout: 525372 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Oct 6 00:33 /home/ubuntu/cephtest 2023-10-06T00:33:02.431 INFO:teuthology.orchestra.run.smithi167.stdout: 917932 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Oct 5 21:06 /home/ubuntu/cephtest/mnt.0 2023-10-06T00:33:02.431 INFO:teuthology.orchestra.run.smithi167.stdout: 917933 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Oct 5 21:07 /home/ubuntu/cephtest/mnt.0/client.0 2023-10-06T00:33:02.431 INFO:teuthology.orchestra.run.smithi167.stdout: 917974 4 drwxrwxr-x 2 ubuntu ubuntu 4096 Oct 5 21:07 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-10-06T00:33:02.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T00:33:02.432 INFO:teuthology.orchestra.run.smithi167.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2023-10-06T00:33:02.432 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_487ffcb5ddb60d29c5688d1a457e812b1b697b90/qa/tasks/ceph.py", line 1907, in task osd_scrub_pgs(ctx, config) File "/home/teuthworker/src/github.com_ceph_ceph-c_487ffcb5ddb60d29c5688d1a457e812b1b697b90/qa/tasks/ceph.py", line 1302, in osd_scrub_pgs raise RuntimeError('Exiting scrub checking -- not all pgs scrubbed.') RuntimeError: Exiting scrub checking -- not all pgs scrubbed. During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi167 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2023-10-06T00:33:02.443 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-06T00:33:02.453 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-06T00:33:02.463 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: RuntimeError: Exiting scrub checking -- not all pgs scrubbed. 2023-10-06T00:33:02.464 INFO:teuthology.nuke:Checking targets against current locks 2023-10-06T00:33:02.491 DEBUG:teuthology.nuke:shortname: smithi167 2023-10-06T00:33:02.492 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T00:33:02.514 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi167.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_14:31:59-rbd-wip-yuri2-testing-2023-10-04-1333-quincy-distro-default-smithi/7411791', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-05 19:54:50.268623', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBIRuzAhk+yMzyhTJFKgQFK82kkX2FDeRQJVhUXDDAz8byAnwnstl+xvDmRzlH0emr8zzGNpH4zXzX5NUQczVBk='} 2023-10-06T00:33:02.565 INFO:teuthology.orchestra.console.smithi167:Power off 2023-10-06T00:33:02.565 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T00:33:02.588 DEBUG:teuthology.orchestra.console.smithi167:power off output: Chassis Power Control: Down/Off 2023-10-06T00:33:02.589 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T00:33:02.612 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is on 2023-10-06T00:33:06.613 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T00:33:06.723 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is on 2023-10-06T00:33:10.727 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T00:33:10.837 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is off 2023-10-06T00:33:10.937 INFO:teuthology.orchestra.console.smithi167:Power off completed 2023-10-06T00:33:11.043 INFO:teuthology.run:Summary data: description: rbd/cli_v1/{base/install clusters/{fixed-1 openstack} conf/{disable-pool-app} features/format-1 msgr-failures/few objectstore/bluestore-comp-zlib pool/small-cache-pool supported-random-distro$/{centos_8} workloads/rbd_cli_generic} duration: 15379.127359867096 failure_reason: 'Command failed (workunit test rbd/cli_generic.sh) on smithi167 with status 124: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=487ffcb5ddb60d29c5688d1a457e812b1b697b90 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/cli_generic.sh''' flavor: default owner: scheduled_yuriw@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=0bdd485f18fc4b7199f1461fe7f6ee79 status: fail success: false 2023-10-06T00:33:11.043 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T00:33:11.116 INFO:teuthology.run:FAIL